Cloudflare Upgrades AI Search to Index Custom Data

Cloudflare has upgraded its AI Search service, a built-in retrieval engine designed to give AI agents and applications a ready-to-use search system over custom data. Traditionally, building a search pipeline requires assembling disparate components like crawlers, parsers, embedding models, vector databases, and search APIs. Cloudflare AI Search consolidates these existing primitives—including Workers AI, AI Gateway, Vectorize, R2, and Browser Run—into a single, automated end-to-end pipeline.

The updated service introduces a discover mode that automatically indexes websites even if they lack a published sitemap. Developers can initialize a search instance using a single command, such as npx wrangler ai-search create cloudflare-community --source https://community.cloudflare.com. The engine can index and search across multiple sources—including Astro, Vite, Hono, Replicate, and Cloudflare's own developer documentation—as a unified corpus. For websites built on the open-source EmDash content management system, integration is simplified via a dedicated plugin.

To deploy the search capability, developers can use a Cloudflare Worker or expose public, unauthenticated /mcp and /search endpoints. During its current beta phase, AI Search is entirely free to use. Once the service reaches general availability, Cloudflare plans to offer a pricing model designed to be "predictable and scalable," as the company emphasizes. Under this structure, embedding and re-ranking tasks will remain free when using default models or select options from the Workers AI catalog, while answer generation and query rewriting will be billed based on active model usage.