UK publishers pool traffic data to spot online scrapers

A free service from NLA Media Access combines traffic data from participating publishers to identify organisations scraping their content. Its aim is to give publishers stronger evidence to seek a licence or take legal action, rather than relying on one site’s traffic patterns alone.

Watch Desk analysis

What happened

The service, called the Collective Intelligence Advisory Service, looks for activity that appears ordinary on one publisher’s site but suspicious across several. NLA chief executive Henry Jones gave the example of a visitor reading hundreds of articles across multiple publishers: individually, that could look like an unusually keen reader; taken together, it may point to a scraper.

Press Gazette reported the launch on 29 October. NLA says a pilot involving Newsquest and three national titles identified organisations that had assumed they could take content freely. Some asked for licence terms after being challenged. The organisation expects a collective case within 18 months, but that is a forecast, not a filed lawsuit. Read the report.

Why it matters

Scrapers can disguise automated activity as ordinary browsing, and robots.txt is a voluntary request, not a magic forcefield. By pooling signals across publishers, the NLA is trying to make that behaviour easier to document and harder to dismiss. Its existing Text Tracker has removed more than 150,000 infringing items this year, according to Press Gazette.

That gives smaller publishers a possible route to act together instead of pursuing costly cases one by one. It also lands amid disputes over how AI companies obtain news content. The service is described as targeting scrapers generally, though, not specifically AI firms; the distinction matters when the eventual legal claims are tested.

Our read

The interesting move is collective evidence, not a promise that every suspicious browsing pattern proves infringement. A reader can visit hundreds of articles for perfectly innocent reasons; the NLA’s case will depend on what its data can establish about the organisation and conduct behind the traffic. Still, a shared detection service may give publishers leverage they would struggle to build alone. The proof will be in the cases, not the dashboard.

What to watch

• Whether the NLA brings the collective case it expects within 18 months.
• What evidence the service shares with publishers and prospective licensees.
• Whether identified organisations are shown to have taken content unlawfully, rather than merely generating unusual traffic.

Discussion spark: Should publishers be able to pursue suspected scrapers collectively on shared traffic evidence, or does that risk treating unusual reading patterns as proof of infringement?

Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.