Root Cause: Months-Old Bug in Networking Gateway Update
Microsoft has confirmed that a software defect introduced during a July update to its Microsoft 365 networking gateway servers was the primary catalyst for the global service disruption that began on November 12, 2026. The bug remained dormant for more than three months before specific traffic conditions activated it, causing a cascading failure across the gateway fleet.
According to a preliminary post-incident report published in the Microsoft 365 Admin Center, the update contained a contention issue that could overwhelm servers under certain load patterns, causing them to stop responding entirely. Microsoft acknowledged that its pre-deployment load testing failed to detect the latent defect because the testing environment used less powerful hardware with fewer cores than the production servers.
"Our normal load testing did not detect the latent issue because the traffic mix and SKUs used in our labs," Microsoft explained in its report. "The contention issue is worsened on machines with more cores and newer hardware; whereas our load testing tended to focus on weaker hardware with fewer cores."
Trigger Events: Teams Failover and Reduced Capacity
The bug was ultimately triggered approximately 15 hours before the main outage when Microsoft Teams automatically failed over to the networking infrastructure used by Exchange Online. This failover significantly increased traffic on the gateway servers. Compounding the situation, nearly 14% of the gateway's server capacity was already offline due to an unrelated issue.
While the infrastructure had sufficient headroom to absorb either the traffic increase or the capacity reduction individually, the combination of both factors created the precise conditions needed to activate the software bug. Microsoft Teams had also experienced a separate outage earlier on November 12, though the company has not confirmed whether the two incidents are related.
Cascading Failure Spreads Worldwide
Once the bug was triggered, affected servers became unresponsive. Microsoft's traffic management systems automatically redirected users to remaining healthy servers, but the sudden influx of redirected traffic quickly overwhelmed those servers as well, causing them to fail in turn. Within two hours, more than 80% of the gateway server fleet was unresponsive.
Safeguards designed to prevent problems from spreading between regions failed to contain the cascade, allowing the disruption to propagate globally. The outage affected Microsoft 365 services worldwide, including Outlook, Exchange Online, Teams, and Copilot, leaving users unable to access email and causing significant message delivery delays.
Service Recovery and Differential Impact
The incident began at 11:20 AM ET on Thursday, November 12, and continued until 11:10 AM ET on Friday, November 13 — a duration of nearly 24 hours. Recovery proved difficult because restarting affected servers caused them to be immediately overwhelmed again by backed-up traffic queues.
While services such as Teams, OneDrive, and SharePoint could switch to alternative infrastructure and recover earlier, Exchange Online lacked the same failover capability, leaving mailbox access impaired for the duration of the incident. After identifying the root cause, Microsoft engineers developed and deployed a fix across affected servers, allowing services to stabilize throughout Friday.
Remediation and Executive Response
Microsoft confirmed that active customer impact had ended by 11:10 AM ET on November 13. The company stated it has since increased gateway capacity, improved monitoring systems, and added safeguards to prevent traffic redirection from triggering similar cascading failures in the future.
In a separate statement on X, Charles Lamanna, Microsoft's executive vice president of Copilot, Agents, and Platform, apologized for the disruption: "We are deeply sorry for the impact to all our customers and recognize the impact that this has on their normal business & operations."
Ongoing Investigation
Microsoft indicated its investigation is continuing, with a final post-incident report expected later this week. The incident highlights the risks of latent software defects that only manifest under specific production hardware configurations and traffic patterns not replicated in test environments.
Source: BleepingComputer