CenturyLink, a US Internet service provider, experienced a significant technical breakdown on Sunday (Aug 30) following a network misconfiguration in one of its data centres.

The error spread outward from the company's network mainly because of the technical nature of the outage that involves both firewall and BGP routing. It even affected other Internet service providers, resulting in several more businesses experiencing communication issues.

OpenDNS, NameCheap, Imperva, Duo Security, Hulu, Reddit, Discord, Steam, Microsoft (Xbox Live), Blizzard, EA, Twitter, Amazon, and many more are on the list big names in the tech sector who had services going down today amid the CenturyLink outage.

Cloudflare, which has also been seriously affected today, said the CenturyLink Outage has resulted in a 3.5 per cent decrease in global Internet traffic, making this one of the largest internet outages ever reported.

"This was a significant global Internet outage," said Matthew Prince, co-founder & CEO of Cloudflare, in his analysis of the outage.

Outage Cause: Erroneous Flowspec Notification

The problem originated from CenturyLink's data centre in Mississauga, a town near Ontario, Canada, according to a status page on CenturyLink.

The company blames the incident on erroneous Flowspec notification. Flowspec is a BGP protocol extension that enables businesses to spread firewall rules across their network using BGP routes. Typically, flowspec notifications are applied to deal with security threats like DDoS Attacks or BGP hijacks or as it enables businesses to adjust their whole network in seconds to respond and prevent attacks.

Today, however, CenturyLink said its data centre at Mississauga has sent out an erroneous Flowspec notification that effectively mitigated rooting of their BGP routes.


 

BGP routes are a kind of message that internet companies transmit amongst themselves. They tell any internet provider which portion of IP addresses is on their network. They act like the glue which holds up the internet.

Unfortunately, as the erroneous Flowspec order from CenturyLink caused some of the routers within its network to break down, some of which even started announcing flawed BGP routes to other neighbouring internet service called "Tier 1."

Consequently, it brought a domino-like impact on other networks.

Outage status

By taking the unusual step of asking all other Tier 1 internet providers to de-peer, CenturyLink solved the problem and avoided all traffic from its network. Companies seldom make such decisions, since this leads to a complete loss of connection for every customer.

Overall, from about 12:13 UTC to 18:58 UTC, CenturyLink was forced to reset every equipment and begin with clean BGP routing tables. It took nearly seven hours to complete the process.


ⓒ 2024 TECHTIMES.com All rights reserved. Do not reproduce without permission.
* This is a contributed article and this content does not necessarily represent the views of techtimes.com
Join the Discussion