Microsoft builds synthetic workplaces to test Copilot without customer data

Microsoft says it has built a system for testing Copilot against synthetic workplaces, giving teams realistic enterprise scenarios without using customer data. The company says the platform has become its first standardised evaluation baseline for Copilot, with reported gains in data quality and faster dataset creation.

Microsoft AI Watch analysis

What happened

The Microsoft Research post describes three simulated environments: pharmaceutical, capital markets, and IT and software. They contain invented users, communications, organisational structures and work histories, drawing on public sources such as occupational data, job postings and SEC filings.

Microsoft says an automated pipeline turns generic workplace tasks into industry-specific questions grounded in those synthetic environments, then checks the questions for accuracy and uniqueness. The company reports a 39% improvement against its initial approach on an internal composite measure of synthetic-data quality. It also estimates that automation saves about $100,000 per evaluation dataset and $300,000 across the project.

Key findings

• No customer data in the pipeline
  Microsoft says all environment content is synthetic, with automated safety checks and legal review of public data sources.
• Three industry settings
  The environments cover pharmaceutical work, capital markets, and IT and software, with simulated histories and workflows.
• Faster, repeatable evaluation
  Microsoft says dataset creation moved from days of manual work to automated runs that can be reproduced from the same configuration.

Why it matters

Enterprise assistants are meant to work across meetings, email, documents and specialised workflows. Testing them on toy examples can miss the organisational context that makes a question hard. Synthetic workplaces offer a way to test more realistic scenarios without putting confidential company records into an evaluation pipeline.

That is a useful direction, but the numbers here are Microsoft’s own internal results. A convincing simulation is not, by itself, proof that a Copilot response will be accurate in a real organisation. The quality of the synthetic environments and the tests built from them still matters rather a lot.

Our read

This is a sensible answer to a genuine testing problem: make enterprise evaluations richer without turning customer data into test material. The useful next step is evidence that these synthetic tests predict real-world performance, not just that they are cheaper and easier to run. Microsoft has built the test track; now the question is how closely it resembles the road.

What to watch

• Whether Microsoft publishes more detail on how its internal quality scores are calculated.
• Whether the synthetic tests are shown to predict performance on real enterprise tasks.
• Whether teams beyond Microsoft adopt the platform or its evaluation baseline.

Discussion spark: Can synthetic workplace data provide a credible test of enterprise AI without real customer records, or should evaluations have to prove themselves against real-world use?

not affiliated with or endorsed by Microsoft