Cohere puts its enterprise retrieval stack into a managed cloud beta

Cohere is opening Compass Cloud to a limited group of enterprise teams, turning its retrieval platform into a managed service for search, RAG and AI agents. The important change is operational: customers can use Cohere’s retrieval pipeline and model inference without building and running the whole machinery themselves.

Cohere Watch analysis

What happened

Cohere says Compass Cloud packages document processing, indexing, hybrid search, reranking and permission controls behind APIs, a Python SDK and a dedicated MCP server. The service can connect to sources including SharePoint, OneDrive and Google Drive, while self-hosted deployments remain available for organisations that cannot send workloads to a SaaS platform.

The company says Compass can enforce document-level permissions during retrieval, apply retention rules and remove deleted content from future synchronisation. Cohere also reports a 14 to 16 point improvement over Azure Search on its internal High Finance benchmark, with nDCG@10 rising from 64.8 to 81.1. That is a company-reported comparison on a Cohere-built benchmark, not a universal verdict on enterprise search.

Teams joining the private beta receive early access, engineering support and a say in the product roadmap. Cohere says it will discuss the future of enterprise search in a live session on 8 October. The announcement was reported by Unite.AI, which identifies its author as an AI-generated strategist whose work is reviewed by the publication’s editorial team.

Why it matters

Retrieval is the bit that supplies an AI system with relevant company information before it generates an answer or takes an action. If the search layer returns stale, irrelevant or over-permissioned material, a polished model can turn bad context into a very confident mess. Enterprise AI, meet the filing cabinet.

Compass Cloud’s pitch is consolidation. Instead of separately operating ingestion, parsing, embeddings, indexing, access control and reranking, teams can configure one managed retrieval service. Its MCP server also lets compatible agents call retrieval tools without a bespoke integration for every framework.

That could reduce the engineering burden, but it also concentrates an important part of the data pipeline in one vendor. Buyers will need to establish how permissions, retention, latency, pricing and portability behave in production, particularly when agents issue many searches and refine their queries over several steps.

Our read

This is more consequential than another chatbot wrapper because retrieval quality and access control are becoming core infrastructure for agentic software. Cohere is making a credible case that enterprises want a managed option, especially when running a full retrieval stack is harder than the demo suggests.

The benchmark result is worth noting, not worshipping. The useful test is whether Compass remains accurate, fast and properly permissioned across a customer’s own documents, languages and awkward legacy systems. Private beta is exactly the stage where those unglamorous details should be forced into daylight.

What to watch

• Independent testing:
  Whether customers or researchers reproduce the reported gains outside Cohere’s High Finance benchmark.
• Permission behaviour:
  Whether document-level access controls hold up across multi-step agent workflows.
• Cloud economics:
  How managed retrieval pricing compares with self-hosting and competing search services.
• Portability:
  Whether the API and MCP support make it genuinely easy to move between models and agent frameworks.

Discussion spark: Should enterprises hand retrieval and permission enforcement to a managed AI vendor for simplicity, or keep that layer in-house even when the operational cost is considerably higher?

not affiliated with or endorsed by Cohere