Microsoft attributes global 365 outage to July software bug and traffic surge
Just the facts

Microsoft attributes global 365 outage to July software bug and traffic surge

Summary

Microsoft said a bug introduced in a July update, combined with increased Teams traffic and reduced server capacity, caused a near-24-hour worldwide outage of Microsoft 365 services on November 12-13.

Microsoft identified a software defect introduced in a July update to its networking gateway servers as the primary cause of the worldwide Microsoft 365 outage that began at 11:20 a.m. ET on Thursday, November 12, and lasted until 11:10 a.m. ET the following day. The bug, undetected during testing that used less powerful hardware, could overwhelm servers under certain traffic conditions.

The issue was triggered after an automatic Teams failover shifted traffic to the infrastructure used by Exchange Online, increasing load on the already strained servers. At the same time, about 14% of the gateway’s capacity was offline due to an unrelated problem, further reducing available resources.

Microsoft reported that more than 80% of the gateway servers became unresponsive, and safeguards intended to contain regional failures did not prevent the cascading outage. While services such as Teams, OneDrive and SharePoint were able to switch to alternative infrastructure, Exchange Online could not, leaving many users unable to access email.

"In less than two hours, the previously unknown defect was encountered across our fleet and rendered them unresponsive. At its lowest, more than 80% of servers were unresponsive," the company said in its preliminary report.

"Our normal load testing did not detect the latent issue because the traffic mix and SKUs used in our labs," Microsoft added, noting that testing focused on weaker hardware.

After the bug was identified, engineers deployed a fix across the affected servers, and services were fully restored by the morning of November 13. Microsoft said it has increased capacity, improved monitoring and added safeguards to prevent similar cascading failures. In a statement on X, Charles Lamanna, executive vice president of Copilot, Agents, and Platform, apologized for the disruption.

The company plans to release a final post-incident report later this week.

Keep reading — for free

Create a free account to follow the news. No card required.