Routine maintenance on Facebook’s network connecting its data centers caused the company’s global system to go down for more than six hours on Monday, the company said on Tuesday.
Downdetector, a web monitoring company, said the outage was the largest it had ever seen. It restricted access to apps for billions of Facebook (FB.O), Instagram, and WhatsApp users, escalating weeks of scrutiny for the $1 trillion company.
A former employee turned whistleblower accused Facebook of putting profits ahead of people’s safety at a U.S. Senate hearing on Tuesday, which the company denies.
Santosh Janardhan, Vice President of Engineering at Facebook, explained in a blog post that the company’s engineers unintentionally disconnected Facebook data centers from the rest of the world.
While users were denied access to one of the world’s most popular messaging apps – WhatsApp has over 2 billion users – employees were also denied access to internal tools.
According to Facebook, the outage knocked out tools that engineers would normally use to investigate and repair such outages, making the task even more difficult.
According to the company, it dispatched a team of engineers to the location of its data centers to debug and restart the systems.
However, due to the high physical and system security in place, it took the company longer to get engineers inside to work on the servers.
A Facebook VP Refutes A Report That Claims The Network Is Aware Of Various Issues It Is Failing To AddressAccording to Facebook, the program audit tool had a bug and failed to stop the command that caused the outage.
Janardhan wrote, “Every failure like this is an opportunity to learn and improve.” “From now on, our job is to… make sure that events like this happen as infrequently as possible.”
Read on Reuters