Microsoft attributes global Outlook outage to July software defect and server capacity issues
Microsoft said a hidden software defect introduced in July, combined with a traffic shift and reduced server capacity, caused more than 80% of its networking gateway servers to fail during last week's global outage.
Microsoft has traced last week's worldwide outage of Outlook and other Microsoft 365 services to a software defect introduced in July, noting that at the height of the incident over 80% of servers supporting its networking gateway became unresponsive. Charles Lamanna, the company's Executive Vice President of Copilot, Agents and Platform, said three factors converged to trigger the failure: the latent defect, a shift in Microsoft Teams traffic to the primary gateway, and reduced server capacity.
"We are deeply sorry for the impact to all our customers and recognise the impact that this has on their normal business and operations," Lamanna posted on X.
The defect, embedded in a fix for the networking gateway servers, remained undiscovered during load testing because the tests emphasized weaker hardware, while the issue affected newer, higher-core machines. About 15 hours before the outage, Teams traffic moved from an alternative gateway to the primary gateway that also handles Exchange Online, increasing load and altering traffic patterns. At the same time, roughly 14% of the gateway's server capacity was already offline due to an unrelated problem.
The combination of heightened traffic and diminished capacity overwhelmed the servers, causing requests to pile up until the servers stopped responding during peak U.S. traffic hours. Microsoft's traffic management system redirected users to the remaining functional servers, which further strained them and led to a cascading failure across the fleet.
"When servers lose connectivity, our traffic management systems automatically route users to other healthy endpoints. This happens often and is critical to avoiding any interruption... However, the redirected traffic increased the likelihood for the same problem to occur on the remaining endpoints," Lamanna explained.
Safeguards meant to isolate parts of the gateway failed, allowing the disruption to spread globally. While services such as Teams, OneDrive, SharePoint and Office apps were restored earlier by moving away from the faulty gateway, Exchange Online could not be shifted and remained affected. Recovery was slowed because servers brought back online were immediately overwhelmed by backlogged traffic.
Engineers later deployed an update to revert the code, taking servers offline in batches, applying the fix, and gradually restoring service. Lamanna said Microsoft will release a detailed post-incident review later this week.
The outage impacted Indian airlines, including IndiGo, Air India and Akasa Air, which reported disruptions to email-based customer support, and financial services firms that struggled to access research reports and client communications. At its peak, the incident generated more than 1.3 lakh reports of Outlook problems on Downdetector, affecting users in 72-74 countries.