Meta says Andromeda now retrieves more than a billion ads a second

Meta says its upgraded Andromeda system can score eligible ads at more than one billion per second in production, bringing near-ranking-level model performance to the retrieval stage. The engineering work matters because it aims to handle a vastly larger pool of ads without a matching leap in inference cost.

Meta AI Watch analysis

What happened

Andromeda is Meta’s machine-learning system for personalised ad retrieval: it narrows a large pool of eligible ads to candidates for a later, more detailed ranking stage. In a new engineering post, Meta says the latest version was designed jointly across the model, software system and GPU hardware, and now processes more than one billion ads per second in production, a 100-fold increase over its 2024 system.

The company describes several changes behind that claim:

– Mixture-of-experts retrieval Specialised user experts are selected dynamically, bringing retrieval closer to ranking-level performance, Meta says, at about 300 times fewer inference FLOPs than its 2024 retrieval model. – More contextual ad candidates The candidate-ad model incorporates real-time serving context, so later interaction models receive context-aware representations. – A fused GPU pipeline Meta says a single GPU kernel combines model and system work that previously moved data across separate stages. – Request logic beside GPU work The new design co-locates per-user request logic with per-ad GPU kernels in one GPU box, addressing data movement and network bottlenecks. – Higher single-GPU throughput Meta reports model inference throughput above 10 billion ads per second on one NVIDIA GH200 GPU. That is a model-throughput figure, distinct from the system’s reported production rate.

Meta also says Andromeda has delivered a 2.4% increase in Facebook conversions since 2025. That is the company’s reported outcome, not an independently established measure.

Why it matters

Ad retrieval has to sift through an expanding pool quickly before a separate ranking stage makes finer choices. Meta says generative AI creative tools have helped advertisers produce hundreds of millions of new ads, increasing the pool the system must handle. If the reported gains hold, Andromeda’s redesign shows how model architecture and GPU-level engineering can jointly raise throughput while keeping inference costs in check.

For advertisers, Meta says the system can match ads across a larger candidate pool within the same latency budget. For users, the system is part of how personalised ads are selected, making its design consequential even if most people will never see the machinery behind their feed.

Our read

The striking part is not just a very large throughput number. It is the attempt to make a more capable retrieval model practical by redesigning the whole path from user and ad representations to GPU execution. Meta’s account gives useful technical detail, but the conversion figure and performance comparisons remain company-reported. The next test is whether those gains persist as the ad pool and models keep growing.

What to watch

• Whether Meta publishes more detail on how it measures the reported conversion increase.
• How Andromeda’s throughput and inference costs change as the candidate pool grows.
• Whether the co-designed approach brings similar gains to other large-scale recommendation systems.

Discussion spark: When a platform says a more powerful ad-retrieval system improves performance, should it also disclose more about how that changes the adverts people are shown?

not affiliated with, endorsed by, or operated by Meta