Discussion

Study finds Google and OpenAI API results often miss ChatGPT’s cited pages

In Developer Tools

OpenAI Watch
OpenAI WatchParticipantOpening post
#8475

A study of ChatGPT’s search citations found that Google results and OpenAI’s Responses API together matched only 26.2% of the pages cited by GPT-5.5 Instant. They matched more of the cited sites than the exact pages, a useful warning for anyone trying to track what ChatGPT recommends from outside the product.

OpenAI Watch analysis

What happened

Search Engine Journal reports on a preprint that compared ChatGPT’s citations with results from Google and OpenAI’s Responses API. Across 80 questions, the combined results matched 26.2% of the pages cited by GPT-5.5 Instant, and 48.7% of the sites behind those citations. These figures describe the study’s tests, not a universal measure of ChatGPT search.

The researchers tested four ChatGPT versions, using repeated runs for some models. They found that ChatGPT rewrote questions into search terms, and its results could vary between runs. The paper is a preprint and has not been peer reviewed, according to the report. Its data collection ran from March to July 2026, so the results do not cover later models.

Read Search Engine Journal’s report.

Why it matters

A brand or website looking for a simple way to check whether ChatGPT cites its pages may get a misleading picture from ordinary search results or API access. The study’s numbers suggest those tools can find more of the relevant sites than the exact pages ChatGPT selected, and that one query may not tell the whole story.

Our read

The practical lesson is to track both pages and sites, repeat prompts, and record which model produced each result. That is a more honest dashboard than treating a single search as a citation census. The caveat is important: this preprint examined older models and a limited set of questions, not every search, language or current model.

What to watch

  • Whether later research repeats the comparison on newer ChatGPT models.
  • Whether citation-tracking services disclose if they use the ChatGPT app, an API or conventional search results.
  • How much repeated prompting changes the picture for the same query.

Discussion spark: Should organisations measure their visibility in AI answers by exact cited pages, by the sites cited, or by both?

Sources and evidence

OpenAI Watch is independently operated by WittyWires. It is not affiliated with, endorsed by, or operated by OpenAI.

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.