Salesforce is using Amazon SageMaker AI’s cross-region inference routing to shift model requests towards capacity that can serve them, while keeping requests with data-residency requirements inside their home region. The setup offers a practical look at how a company serving AI models across 30-plus regions handles capacity spikes without building every failover rule itself.
Salesforce Watch analysis
What happened
Salesforce’s Agentforce Foundation team serves in-house and open models across more than 30 regions and hundreds of AWS accounts. In an AWS blog post, the team describes replacing its own gateway’s basic health checks and region-switching logic with SageMaker’s inference router.
The router prefers local endpoints, then shifts a growing share of new requests towards healthy destinations when recent request outcomes indicate a problem. Salesforce pins requests marked with a residency requirement to a routing target containing only a local endpoint. It logs which endpoint served each request and why, giving teams a record to investigate traffic shifts.
Why it matters
The change tackles a familiar infrastructure squeeze: spare capacity in one region does not help much when another is running short. Routing can spread requests across capacity Salesforce has already deployed, but it does not deploy models or add instances. The practical gain is managed routing and clearer visibility, not a new supply of compute appearing by magic.
For operators, the residency rule is especially important. Salesforce describes a way to use cross-region capacity for eligible requests while keeping residency-tagged traffic within its region. The post is a company implementation account, not an independent performance benchmark.
Our read
This is a useful example of AI infrastructure engineering where the interesting part is not a new model, but what happens when serving capacity gets lopsided. Moving health assessment and failover into a managed routing layer may reduce bespoke gateway logic; the local-only residency path shows how that flexibility can have a boundary. Read AWS’s account of Salesforce’s setup for the implementation details.
What to watch
- Whether Salesforce shares measured changes in availability, latency or operating effort.
- How the router behaves as traffic shifts between regions during real capacity pressure.
- Whether the residency controls and routing telemetry meet the needs of other teams running models across regions.
Discussion spark: When AI serving capacity runs short, should teams rely on managed cross-region routing, or keep failover logic in their own systems for tighter control?
Sources and evidence
- How Salesforce is advancing global-scale model serving with cross-Region inference routing in Amazon SageMaker AI (18 November 2026, 20:13 UTC)
Independent WittyWires tracker for public updates about Salesforce. Not affiliated with or endorsed by Salesforce; this is not an official account.