Last Thursday’s NewsForge report about power grid security prompted an outpouring of response from readers, industry experts, and government officials. Experts hastened to point out that the grid is robust and relatively immune to hacker attack even as attention focused on the role that a suspiciously frozen computer in the control room of grid operator
Dr. Anjan Bose, Distinguished Professor of Power Engineering and a Dean at Washington State University, says, “The fragility of the grid is overrated. It’s still the most sophisticated and reliable power grid in the world.” He points out that the grid, like the Internet, routinely routes around failures. Given that the last major outage was in 1965, Bose thinks the system is pretty good, even though it wasn’t up to handling the events of Aug. 14.
Bose says that the grid is designed to continue operating even if operator control computers were to go down. Prof. B. Don Russell, Regents Professor of Electrical Power Engineering at Texas A&M, agrees with that assessment. “These are old systems; most of them were built before today’s hackers were born,” says Russell. Many systems aren’t really general-purpose computers but consist of 1970s-era controller boards running custom firmware that are hard-wired to circuit breakers and other gear.
Grid operators’ Unix systems are also relatively old, and some are CRT versions of the “minute boards” that hang from control-room walls, according to Russell. These systems show a schematic of the operator’s grid with colored lights or other indicators that display the status of individual components, such as circuit breakers. Operators make changes to the system with a light pen, cursor, or typed commands.
Power-grid control systems are not normally connected to public networks or each other, a point repeatedly emphasized by Bose and Russell. The power grid’s control “networking” consists of 24×7 shifts of operators making long-distance phone calls to each other. “Cyber attack isn’t even on my top 10 list [of threats],” said Russell. Bose offered that it would be much easier to blow up transmission towers and substations than cause an outage by hacking.
Bose and Russell are both members of the National Academy of Engineering, a federally funded group whose charter includes providing independent counsel to the President and Congress on engineering issues.
Even if a hacker did bring down a piece of equipment, the grid would simply route around it. Both Bose and Russell felt that the grid’s “security by antiquity” would make it extremely difficult for hackers to create a major outage.
But transcripts of phone calls released at last Wednesday’s Congressional hearings into the causes of the blackout show that at least one FirstEnergy technician was unable to answer questions being asked by neighboring grid operators and utilities because of a computer problem that kept him from knowing the status of his systems. An earlier report in the Cleveland Plain Dealer quoted two municipal electrical system operators as saying FirstEnergy’s systems were down during the crucial hour leading up to the blackout.
And, during the hearings, Rep. Edward Markey (D-Mass.) is reported to have said that FirstEnergy had been hit by the Blaster worm a few days prior to the shutdown and asked FirstEnergy CEO H. Peter Burg if his company had taken steps to see that computers were protected. Burg replied that its grid-control computers were not susceptible to Microsoft viruses. Burg earlier testified that he believed a wider combination of events not limited to FirstEnergy was the cause of the outage.
Bose and Russell both think that the technician would have been using a Unix-based system like the one Russell described. FirstEnergy spokesperson Ellen Raines confirmed that the Akron control center uses a General Electric energy management system that she believed uses the Unix operating system. This may have significance in light of an article published by black-hat-hacker-turned-security-consultant Kevin Poulsen on Aug. 19.
In that article, Poulsen offers a detailed description of how another Microsoft worm, Slammer, crashed two Unix-based control systems at the Davis-Besse nuclear power plant in Northern Ohio also operated by FirstEnergy. Poulsen reported that FirstEnergy engineers had bridged the nuclear plant’s control network with FirstEnergy’s corporate network — a practice that is increasingly common among utility companies, according to industry and security experts.
The corporate network was firewalled, but an unprotected connection inside the firewall by a contractor allowed Slammer to penetrate, eventually causing high network traffic that overwhelmed the plant’s process control computer and Safety Parameter Display System. The Davis-Besse plant was offline at the time and had redundant backup systems that were not affected.
An advisory released by the Nuclear Regulatory Commission confirmed on Sept. 2 the details first revealed publicly by Poulsen and noted that the systems were unavailable to operators for several hours because of the effects of the Slammer worm.
A reference in last Thursday’s NewsForge story by this writer to earlier wire service reports that “DOE officials” were discounting cyber attack in favor of more mundane grid failure theories was rebuffed by DOE Chief of Staff Kevin Kolevar, who noted that DOE has not ruled out any scenario.
A number of NewsForge readers pointed to the Poulsen article, speculating that a similar event — this time caused by Blaster worm traffic — could conceivably have prevented the Unix-based computers in FirstEnergy’s Akron control room from operating properly, thus robbing FirstEnergy technicians of vital information during a time when it is thought that four of the company’s transmission lines, as well as three other Ohio lines, were failing. FirstEnergy’s Raines said that she believed that the control system’s network was independent of their corporate network.
While this is purely theory at this point, and officials anticipate a lengthy and difficult investigation, a finding that the worm had contributed to the Aug. 14 blackout would demonstrate that the power grid is susceptible to hacker activity, even if it was an indirect consequence. Vulnerabilities in Microsoft products have been cited as a root cause in an already long list of recent disruptions, including shutdowns at a railroad, airline delays, the shutdown of 3,500 police department computers, and disarray at numerous government and corporate offices.
If the Aug. 14 blackout, which is thought to have cost billions in lost productivity, proves also to be on that list, it would bolster the case for using MS-malware-immune Open Source technology to secure the power grid, among other vital systems.
Chris Gulker, a Silicon Valley-based freelance technology writer, has authored more than 130 articles and columns since 1998. He shares an office with 7 computers that mostly work, an Australian Shepherd, and a small gray cat with an attitude.
Category:
- Security
