AI companies are putting prices on the web, but who decides what a page is worth?

The web is acquiring a price tag for AI use. Google, Cloudflare and Microsoft are testing different payment models for content that gets crawled, retrieved or meaningfully contributes to an AI-generated answer, and the differences could decide whether publishers opt in, opt out or accidentally trade search visibility for a seat at the table.

Watch Desk analysis

What happened

Search Engine Journal compares the emerging models. Google’s invitation-only pilot pays when a page is judged to have contributed significantly to an answer across Gemini, AI Overviews and AI Mode. Cloudflare’s Pay Per Crawl is more mechanical: a site sets a price and charges when a crawler accepts payment and receives the page with an HTTP 200 response. Microsoft’s Publisher Content Marketplace is built around publisher-defined licensing and usage terms, with Copilot as its first demand partner.

The useful detail is what remains unsettled. Google does not publicly explain how it measures “significant” contribution. Microsoft promises usage-based reporting without specifying the payable unit. Cloudflare’s newer Pay Per Use experiments leave the trigger dependent on the partner. Read the full comparison from Search Engine Journal for the model-by-model detail.

Key findings

• Google pays for judged contribution
  Its pilot reportedly displays monthly earnings, but the formula for deciding meaningful contribution remains a black box.
• Cloudflare prices the retrieval
  Pay Per Crawl lets a site set a per-crawl price and choose whether individual crawlers may charge, pay or be blocked.
• Microsoft leaves the terms with publishers
  Its marketplace promises licensing control and usage-based reports, while leaving the payment unit unspecified.
• The search trade-off is real
  Charging or blocking search crawlers can affect indexing and visibility, so an AI control is not automatically a harmless switch.
• Outcome-based payment is replacing raw crawl counts
  The comparison highlights a shift from paying for fetching a page towards paying for how that page is used in an answer.

Why it matters

For publishers, this is not merely a question of whether AI companies should pay. It is a question of measurement, control and bargaining power. A site may prefer Google's claimed value-based model, Cloudflare's explicit per-fetch price or Microsoft's licensing approach, but each exposes a different blind spot.

The practical wrinkle is gloriously awkward: the setting that blocks AI training may not be the setting that preserves search traffic. Publishers need to know exactly which crawler is being controlled, what event creates payment and what reporting comes back before agreeing to anything.

Our read

The industry is moving from “please respect our content” to “define the transaction”, which is progress of a sort. The next useful standard is not another grand promise. It is a clear, inspectable record of what was used, why it was valuable and who set the price.

If you run a content site, treat these programmes as commercial infrastructure, not a friendly toggle. Check the crawler rules, the reporting and the search consequences before opting in or blocking anything.

What to watch

• Whether Google publishes a usable definition of “significant contribution”.
• What Microsoft’s first marketplace participants can actually see in usage reports.
• Whether Cloudflare’s Pay Per Use partners disclose rates and payment triggers.
• Whether publishers gain URL-level transparency without damaging ordinary search indexing.

Discussion spark: Which payment model would you trust most as a publisher: Google's contribution-based system, Cloudflare's per-retrieval pricing or Microsoft's publisher-controlled licensing, and what evidence would you demand first?

Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.