November 20, 2003

DOE Releases Blackout Report

Cooper Stevenson writes "The Department of Energy released today their report outlining the causes behind the August 14th electrical blackout that left an estimated 50 million people without power for two days in some areas. You may view the full report here.

On page 22, the report gives us the first glimpse of the two primary reasons for the blackout:

"Subsequent computer failures leading to the loss of situational awareness in FE's control room and the loss of FE transmission lines due to contacts with trees were the most important causes."

The task force reveals further details on page 28:

"Starting around 14:14 EDT, FE's control room operators lost the alarm function that provided audible and visual indications when a significant piece of equipment changed from an acceptable to problematic condition. Shortly thereafter, the EMS system lost a number of its remote control consoles. Next it lost the primary server computer that was hosting the alarm function, and then the backup server such that all functions that were being supported on these servers were stopped at 14:54 EDT."

Translation: The alarm PCs in the control room were under a denial of service attack caused by the "Blaster" worm. They probably looked to the user as if they were running normally as DDOS attacks primarily target the networking subsystem. As the worm spread, it infected and overloaded the Primary Domain Controller (signified as the "primary server computer" in the report) and caused the operating system to either crash or experience an overloaded network subsystem. The Secondary Domain Controller also either crashed or became incapable of communicating on the network as the worm exacts it's spayload. The PDC is further running the Supervisory Control and Data Acquisition (SCADA). It is this application that is responsible for grid monitoring and sending alarm notices to the monitor terminals.

So it is becoming clear that the blackout of last summer was caused by a security flaw in the Microsoft line of server products. If you're not convinced, consider the report's hinting about this on page 94:

"Many malicious code attacks, by their very nature, are unbiased and tend to interfere with operations supported by vulnerable applications. One such incident occurred on January 2003, when the "Slammer" Internet worm took down monitoring computers at FirstEnergy Corporation's idled Davis-Besse nuclear plant. A subsequent report by the North American Electric Reliability Council (NERC) concluded that, although it caused no outages, the infection blocked commands that operated other power utilities."

If there's even a hint of lingering doubt, here's the ComputerWeek article outlining the Blaster worm's role in the blackout shortly after it happened. Ironically, the worm even delayed New York's restoration of power, as is stated in the article:

"A former Bush administration adviser who has consulted with the U.S.Department of Homeland Security on the power grid issue said the Blaster worm also hampered the ability of utilities in the New York region to restore power in a more timely manner because some of those companies were running Windows-based control systems with Port 135 open - the port through which the worm attacked systems."


Click Here!