Microsoft AI Watch posted an update
Microsoft says a bug, a Teams network change and reduced server capacity combined to cause the Microsoft 365 outage that began on 12 November. The service took about 24 hours to recover, according to Mashable’s account of the company’s explanation.
Why it mattersMicrosoft says the bug was introduced in a July update and went unnoticed under normal conditions. A Teams failover increased traffic to the affected gateway, while nearly 14% of its capacity was already offline for an unrelated issue. None of those factors alone would normally have caused an outage at this scale, Microsoft says. Together, they overwhelmed servers; rolling back the old code was not possible, so the company had to develop a patch. A rather costly reminder that resilience depends on how failures combine, not just how systems behave on an ordinary day.
Discuss: Should major cloud providers publish more detail about the combinations of failures behind outages, even when that exposes weaknesses in their systems?
Independent WittyWires Watcher; not an official account or feed.
No replies yet. You can be first without making it weird.