Linux.com

did one power plant ever go down because of internet-security? they go down because of not being able to meet electricity-demand given their limited capacity. And when one goes down, the other must produce more, so they have a chance to go down too and so forth. Excess capacity is needed, but private undertakings want to invest ass less as possible to meet certain demand. i can't imagine power plants network every device to every other device, keeping the door open.

#

"invest ass less as possible" has to be the phrase that pays for today.

Some people have a way with words, and others...no word way with have.

#

Did you even bother to read the article before posting?

Note this part:

> The Slammer worm penetrated the plant's internal network and lodged in an unpatched Windows server. The worm's scanning slowed the internal network to a crawl, eventually crashing the plant's Safety Parameter Display System...

Maybe the plant managed to stay up, but the danger was there.

For you to claim that it's not a problem, is like me saying, "My brakes may have failed, but I managed to steer away from everything until the car came to a halt, so it's not a problem."

Windows should not be used for enterprise level applications. It's not safe.

And neither Windows nor Linux should be used for things like running a nuclear reactor. For that you need a fault-tolerant, mathematically-verified micro-kernel.

One candidate for critical operations is QNX. In the future, however, in line with the article's thesis, an Open Source micro-kernel might be even better. Maybe that's where GNU Hurd will initially find its niche.

#

I'm not aware of the plants design but;

The system lost was titled "Safety Parameter Display System". This sound very much like an alarm annunciator. I'd be shocked if they don't also have a hardware annunciator. Modern plants have provided additional alarm and monitoring systems over their "business" computer networks to extend the information to others not directly involve with the plant's operation. But these are not the primary systems used by operations personnel.

From the description of the problem I am very sure that the plants controls and protections were unaffected and would have operated to protect the equipment if a critical fault had occurred. These controls and protections are required to operate correctly in the event that all remote interfaces are lost.

#

Even if there was no "real" safty risk, the fast that they system was affected at all indicates a problem. The MS series of products is simply not secure enough to put our trust in for any application beyond a word processor. It's sad really.<nobr> <wbr></nobr>... would anyone use a Yugo engine in an ambulance?

#

"The MS series of products is simply not secure enough to put our trust in for any application beyond a word processor"

Are you sure ? look at office security venerability!<nobr> <wbr></nobr>:)

-Manoj

"Sorry for possible bad english I speak perl better."

#

> And neither Windows nor Linux should be used for things like running a nuclear reactor.

Agreed.

> For that you need a fault-tolerant, mathematically-verified micro-kernel.

Disagree.

If you need an OS to run something as mission-critical as a nuclear reactor, you need to buy it from a company that is willing to guarrantee the OS will not fail. And has enough money to be worth the trouble of suing.

MS has the money to meet the second requirement, but the idea of being accountable for their software is entirely alien to their mindset. QNX (which you mention) meets the first requirement, but probably not the second.

And I believe that whoever put that Windows server on such a critical network should have been walked out the door, & their supervisor severly punished, if not fired. The same if it had been a Linux or BSD server. None of these has anything close to the reliability requirements that a power station needs.

Geoff

#

But relying on past causes ONLY to predict future vulnerabilities is no longer adequate. After all, before 9/11 it could have been argued "has a single sky-scraper ever been collapsed by a colliding aircraft?" Let's not dismiss the possibility so lightly -- after all, we in the US now know that there are malevolent people out to exploit ANY vulnerability they find that will harm our society. Sure, the capacity issues must be addressed as well -- but computer security of the control systems should be a concern.

#

Sadly, you are mistaken. I used to think as you do. After all, a power plant is large and complicated. They cost millions of dollars. I protect my home network better than that.

But I recently read a story about a nuclear power plant in the US going down because the monitoring system was overwhelmed by a MS worm.

The control system had absolute no protection and it was on the same network as the corporate servers, office desktops, and had a direct connection to the Internet.

The topology of the company's network did include an external firewall, but there were other Internet connections which basically made it useless.

Of course nobody running such a critical application should rely on the perimeter firewall. There just has to be some security in depth.

The control system should probably be on their own network with only a single controlled connection to the corporate network. That connection could be through something like a proxy rather than a filtering firewall to better control traffic.

It seems likely this article was a reaction to the same event.

#

If you ever had a chance to work in a large industrial type environment, you'd see just how networked everything is. I used to work at a fiber optic cable mfg plant. Each tiny little solenoid, probe, sensor, heater, etc on the manufacturing line was actually a networked device that we plugged into a Programmable Line Controller. The PLCs we used were programmed and continuously controlled with a Windows 2000-based PC. Each of the PC's were in turn connected to the rest of our office network. Now, I was able to access certain office computers via the internet and I'm a moron when it comes to hacking. I couldn't hack my way out of a cornfield. But I could certainly imagine that if you had a little insider knowledge or some hacker-savant knowledge, you could run the extruders from your laptop while sitting on the John in your apartment's bathroom!

who knows how everything else is connected.

#

In the article they stated the effect on a powerstation's network. What do you think the effect of taking down safety monitors on a nuclear plant might do?

Hello are we thinking yet?

#

While it's been a few years since I was last in a nuclear plant, I'd suspect that a nuclear plant would be less vulnerable than most power plants. For one thing, only two new plants have gone on line since 1990 (Source: http://www.nei.org/doc.asp?catnum=3&catid=13)<nobr>.<wbr></nobr> Most, last time I checked, were still using early 80's type computer systems. It's harder to hack a small-and-stupid OS.

    Also, nuclear plants are designed with redundant independent parallel monitoring instrumentation; take down the computer, and you can still use the old analog instrumentation.

    Finally, nuclear plants are designed so that the preferred failure mode when something goes wrong is "...and the reactor shuts itself down." Nuke plant operators are NOT stupid; they know they are dealing with the greatest concentration of power mankind has ever manipulated.

      Computer sabotage MIGHT allow you to take a nuclear power plant off the grid. It would require multiple additional simultaneous failures (accidental or deliberate sabotage) to occur at the same time frame to generate an incident even as interesting as TMI (which had only a minimal radiation release; maximum off-plant individual dosage was under 1 millisievert; normal annual backround was 3).

      Creating a Chernobyl type incident at a US (non-soviet) design power plant would require expert on-site sabotage in force, and probably the use of explosives to breach the concrete containment. You won't do it with just a script and a mouse click.

#

Well, rtai or rt patched Linux (to get real time)certainly would be better than MS.
BPA, for years used carrier current over the tranmission lines for relaying. Way to slow for realtime control. Later came microwave links between power plants/switchyards. I cannot imaging that good engineering practice would place critical operations on the internet; but then some people have rose colored glasses.

#

Well, of course you can't shut a power plant down by messing with Windows, but communication after the meltdown was probably hampered by Blaster, possibly slowing down recovery.

Check out this ComputerWorld article:
http://www.computerworld.com/securitytopics/secur<nobr>i<wbr></nobr> ty/story/0,10801,84519,00.html

Ralf-Peter

#

Oops, trying again:
http://www.computerworld.com/securitytopics/secur<nobr>i<wbr></nobr> ty/story/0,10801,84519,00.html

#

Well, it wasn't me, it was the server.
Delete the space in "/securi ty/" and the URL works.

#

What can I say here ?

#

Did the Northeast power outage really start due to a safety system network getting hit with Slammer? This is the first I have hear that reported, although I have not been following this story closely.

If it is true, there really is a lack of qualified computer professionals working in the industry. A scarry thought.

-Pete

#

IIRC the slammer worm did hit a power plant in January or February of this year and did shut down an Ohio plant's monitoring systems for several hours! Luckily the plant had been offline for some time due to a fist-size hole in one of the reactors.


   

#

> Did the Northeast power outage really start due to a safety system network getting hit with Slammer?

No, that hasn't been proven. The January event (as described by the above poster, and mentioned in the article) and the blackout were two separate incidents.

But Windows has not been ruled out in the blackout. The timing of the recent viruses/worms and the blackout, plus the mystery of the safety mechanisms that failed to operate as they normally do, strikes some people as a suspicious coincidence.

Of course, even if it was Windows, we may never hear about it. I wouldn't be surprised if, at the urging of certain well paid (and well connected) government representatives, the underlying cause is kept secret "for reasons of national security."

#

Well, the author left out a few details here. The monitoring system had a redundant analog backup, which means that even if it was attacked by the worm, the backup was still online, and unaffected by it. Most of the problems lie in the fact that many of the deregulated power companies don't want to throw money at things that don't seem to be a huge threat at the moment. If its working why bother fixing it?

#

> The monitoring system had a redundant analog backup...

Actually, when you think about it, they have _no_ backup system.

That's because they have only one reliable system, which is the _analog_ system.

Since their Windows system is insecure, and thus unreliable, it can't be viewed as either a primary or a backup system.

Therefore, in order to have a backup system, they need to replace the Windows system with something else.

#

Even though DOE and other sources ruled out cyber attack as a cause for this month's blackouts

I don't think they have ruled out anyting. In fact, there was a quote from the transcript from First Energy read on the news last night where the person specifically said they were having "computer problems" when the ____ hit the fan.

#

Great daydream but most of the plants and substations are so old that their basic underlying control system is either numatic or analog.

#

It has nothing to do with the subsystems. It has to do with the monitoring systems that are supposed to react to, and shut down anomalies before the whole thing fails. And they didn't. They didn't because they got Blastered and were responding neither accurately nor in a timely fashion, if at all.

#

I'm in the business and this is probably true. All the old systems put outputs into the 10Mb network that are monitored at the desktop with an MS system, so the operators can have their Outlook. Blaster hampered the recovery as well. Although nuclear operators still have very old pdp-type systems, all engineering experts use MS desktop systems as 'over monitors'.

#

I'm in the business too. In fact I design and install electrical network protection systems and this is wrong. The computer systems at the ISOs (independent system operators) used for SCADA (supervisory control and data acquisition) are not MS Windows based. However, even if the SCADA was windows based, these systems are not used for actual system protections, as the network operators do not have time to respond. Cascade outages happen within in seconds or a part of a second. In any case the protection engineer would not trust the operator to respond in time to protect equipment.

All the network protection systems I'm aware of, which are quite a few, do not have any connections to a network other than for plain status and alarming. In fact, the bulk of these protections are not even digital.

There has been no evidence at all to suggest that any virus impacted the recent blackout.

#

Your claim that Windows cannot be involved is contradicted by the January event, as mentioned in the article:

> Just such a problem surfaced in January at the Davis-Besse nuclear power plant operated by FirstEnergy, the Ohio utility under close scrutiny for its role in the East Coast's largest-ever blackout. The Slammer worm penetrated the plant's internal network and lodged in an unpatched Windows server. The worm's scanning slowed the internal network to a crawl, eventually crashing the plant's Safety Parameter Display System, according to reports.

See the article for the links.

However, I am willing to concede the possibility that the systems may vary from state to state, and that some states have utility managers who are competent enough to keep Windows off of their networks.

#

See rely <A HREF="http://newsforge.com/comments.pl?sid=32904&threshold=0&commentsort=0&mode=thread&tid=&pid=68734#68740" TITLE="newsforge.com">here</a newsforge.com>

     

some states have utility managers who are competent enough to keep Windows off of their networks

Personally, I don't trust the utility managers, but I do trust the utility engineers. I trust then to keep the "business" side of the house from affecting the "operations" side. Otherwise they will quickly lose their jobs. Normally there is complete isolation of the "operations" network from the "business" network. However, in some cases there is information exported to the "business" computer networks. Some may get alarmed that there is some of the "operational" information available on the "business" networks or even on the Internet (<A HREF="http://www.caiso.com/SystemStatus.html" TITLE="caiso.com">for example</a caiso.com>). However, utility engineers are very careful that these interfaces are unidirectional. For instance it's typical to disconnect the TX conductor altogether to create a physical disconnect when exporting operational information.

#

As it was with the Romans...

The fabric of our civilization has become and is becoming so complex that pin pricks in the fabric
can cause the whole of the fabric to collapse.

The economic effects of the black out cost billions. The attack on the world trade center cost billions. Both of these have devastating effects on the economy that our civilization is built on.

So those survival nuts were probably right in preparing for the total collapse of their world.
The Romans could not see this happening to them.

That guy the UNA-BOMBER may be right.

#

It's not that it's too complex, it's that the people in charge are too retarted. A Linux-based, more distributed power grid would not have as many problems.

Also, note that at least one utility was able to sever their connection to other utilities and keep providing power to their own individual customers.

The problem is not that it is too complex, it's that it's too stupidly managed. Notice how when things started going wrong, no one said "How do we fix this?". No one said "Let's build a better power grid so this doesn't happen again." Everyone said "This is someone's fault, dammit! Let's go find someone to blame! It sure isn't my fault!"

Some people say that computers are getting too complex, and that's why they crash all the time, have all these viruses. I have a Linux box. It crashed once, while I was using some extremely experimental software and playing a hardcore video game. Linux is very complex, and very stable.

The Romans failed, probably for the same reason -- stupid maniacs in charge.

#

"We have no clue. Our computer is giving us fits, too," replied a FirstEnergy technician identified as Jerry Snickey. "We don't even know the status of some of the stuff (power fluctuations) around us."

A short time later, a technician at the Midwest Independent Transmission System Operators, the group that monitors the Midwest power grid, expressed frustration with FirstEnergy's failure to diagnose the problems erupting in their power system.

#

"The guy told us he didn't know what was wrong, because his computer was down," Dupee said.

-- The Plain Dealer 08/28/03

#

Living right on the edge of the Great Blackout of '03 (a few million people in Michigan did loose power, although you'd never know it from all the east and west coast pundits), and living about an hour's drive north of Besse-Davis, I pay attention to this stuff. (B.T.W.: the organization within Michigan that coordinates electric power distribution between DTE, and Consumers Power is located in my county.)

My observations are:

(1) Besse-Davis was vulnerable during an unusual circumstance<nobr> <wbr></nobr>.. where the reactor was being repaired (the hole in the lid).
This isn't to offer an excuse, but rather to wave a warning flag about how unusual circumstances, repairs, upgrades, etc. can defeat normal security precautions.

(2) DTE, et. al. in Michigan are doing a better job of ducking blame for the blackout than First Energy is.

(3) Legacy IT systems, that simply aren't capable of connecting to the internet are awfully appealing.

(4) Open-Source for large power grid management system is probably not feasible, because the skill-sets are rare, the equipment is probably unique, and the "market" is small (if not unimportant.) Perhaps a government funded project that made sure that all the work was made "open" would be able to produce something, I think having a diversity of systems (even if they are closed and propietary) is much better than a Windows-like mono-culture like blessing an "official" grid management system would produce (even if it was open-source.)

(5) Diversity of grids is important. I recall seeing a map of grids in North America a few years ago (it was more a map of coverage areas), and I was alarmed by the lack of diversity. I think there should be more independant grids, covering, at most a few states, without significant interdependancies such as has evolved east of the Mississippi River.

#

Open-Source for large power grid management system is probably not feasible, because the skill-sets are rare, the equipment is probably unique, and the "market" is small (if not unimportant.)

Actually, this is an excellent place for Open Source. The "market" is irrelevant to Open Source; we're not selling it here. The best solution would be to have the NRC (Nuclear Regulatory Committee) and others work together to create Power-Grade Linux systems, specialized to the system. This is the power of Free Software--it can be modified as needed. They can use commodity software--Linux kernel, GNU utils, Qt Embedded, framebuffers, etc. to create a highly specialized, competely transparent appliance. If they want to keep the mods secret for national security, the NRC can say "we'll give you this box; it's ours, not yours," and they need not give the plants the source, since they're not distributing it.

Really, I don't see why more branches of the government don't get on the Open Source bandwagon; the ability to modify the software you use to completely fit your situation is extremely powerful.

#

There's a great imaginary story that never happened.. Seems a giant utility had a complex Unix firewall that was set up by bright people who were then forced to leave. Maintenance was then taken over by in-house MS script-types who just kept feeding in configuration changes. Being a good unix machine, it worked for years without anybody intelligent touching it.

Come the blackout, the backup batteries ran out and the poor thing bloofed. MS people had not hardened the configuration changes, nor kept any backups! They just fired it on when the power came back. Lo and behold it reverted to the years-old defaults, which was essentially naked! In came all the fun blasters! Took them a while to throw the big switch and cut everybody off from the outside, for days, while they recreated the configurations. Perhaps it's an MS firewall now.

#

Your very lives may depend on Windows security, or rather insecurity... My my, you Americans are a trusting lot!!

#

Although the issues the article raises are real the utilities are not stupid. They are very aware of Internet security issues and do take very reasonable steps to make sure access is limited.

One issue the article clearly misses is the distinction between the "control and protection systems" and "alarm and monitoring systems". Control and protection systems do not normally have any remote access even to on-site network operators. A protection engineer would never trust an operator to protect equipment. Even if the operator does have access to these controls the interface is limited to changing set points and switching.

The article's statement "The situation is so bad, experts say, that bored script kiddies could soon be knocking out power stations as easily as they concoct viruses from toolkits available on the Web" is wrong. Under investment in the power system infrastructure has caused problems but this is not related to Internet security issues. In fact, the systems that have interfaces to a utility's commercial computer networks are some of the most modern systems in use and have been secured properly. I think that Brian Ahern is overstating the problem to help sell his products. And I don't think most electric utility engineers agree with his companies products that extend operator interfaces over commercial computer networks. In most cases there is complete seperation between the utilities computer networks that are used for operations and those that are used for commercial purposes. I work at a utility in controls and protections and I am not aware of any control or protection that has a remote interface on the utility's commercial computer network.

#

Your idea that the power companies are intelligent does not match my life experience at all. My former father in law droped out of school after the third grade. He ultimately became the chief watch engineer for a number of power plants. When nuclear plants came on line he was assigned to Floridas Turkey Point nuclear plant as the watch engineer and also to the nuclear plant at Indian River. He begged and begged to be put back into conventional plants as he knew he could not operate the nukes safely. At the time they were still using slide rules and doing calculations during urgent situations. He had been sent to training classes but was the first to admit that his grasp of the subjects was thin, that he was slow, and that he could not do the work under pressure. The guy operated nukes for at least five years that I know of. It scared the hell out of him.

#

Hmmm... is his name "Homer" by any chance?<nobr> <wbr></nobr>:-)

#

I wrote the initial thread and have about 30 years of utility work experience. I do agree with you that utilities should not be running nuclear power plants. Utility companies are to profit orientated.

#

> Although the issues the article raises are real
> the utilities are not stupid. They are very
> aware of Internet security issues and do take
> very reasonable steps to make sure access is
> limited.

would these be the same corporations that are aware of the health risks in dumping nuclear and other contaminated waste into rivers and take "very reasonable" steps to ensure they aren't caught very often?

#

If it is true that the power blackout was indeed precipitated by a worm, should not Microsoft be held liable for paying the billions of expense dollars incurred as a result of their pathetic security?

#

The EULA in every MS product holds them blameless for anything related to thier software...

#

Dude. <A HREF="http://www.perl.com/language/misc/virus.html" TITLE="perl.com">It's "viruses", not "virii"</a perl.com>. Shouldn't you spell-check your article before posting it? Misspelling affects your credibility.

#

Does it really matter. Everybody understands what both virii and viruses mean. The point of language is to convey meaning - not to see who can piss further.

#

If you want to get technical, viruses probably isn't the correct term either.
Very few of the malignant programs today are actually "viruses".
Most are worms, trojan horses, or a combination.
It's just easy to call them all viruses so that's what most people do.

#

It's interesting that the m$ comments are suceeded with a m$ ad.

#

The problems the utilities are having are the result of basic design flaws in their networks. It doesn't matter whether the shoddy software comes from Microsoft, <A HREF="http://www.redhat.com/solutions/security/news/" TITLE="redhat.com">Red Hat</a redhat.com>, or anybody else. What matters is that idiot network engineers weren't willing to grow the balls to tell their bosses to separate the business networks from the monitoring networks.

Trust is viral. When you start connecting insecure, semi-trusted business networks (whether they be hosting UNIX, Linux, Microsoft, or VAX machines) with mission critical systems, you're asking for trouble.

Open source will not cure this industry's security flaws. But common sense will. Your (most likely unpatched) mission critical system is only as safe as the least trustworthy host that can talk to it.

#

There is the way it should be, and the way it is. If you are a home user, you can go get your coffee, download your MS patches, and away you go. If your a huge company with thousands of machines, things get a little more complicated. Ever heard of change manangement? Testing the patches against all your application software before pushing it across the network? This is a extrememly tedious, but necessary time consuming task. "We had the patch out two weeks ago". That's barley time to allocate man hours to testing the patch. If you don't believe me, ask Microsoft, they have gone down from not patching thier own machines.

#

please do not use virii

http://www.perl.com/language/misc/virus.html

#

single 'form', nice mistake<nobr> <wbr></nobr>:)

#

Ahern is mainly wrong about packet filters. IP isn't a realtime protocol. Firewalls are just routers with teeth.

It could be netware, but it suffers the simular issues. Regardless, should be zero Internet facing hosts that are attached to a control system. While amazingly conveniant, it is giving away the keys to the castle.

There should be an entirely seperate physical network for this. Without any "firewalls". if traffic peaks on the untrusted size, the control side is unaffected.

I just see a major contradictions in this mindset, they aren't willing to setup secuity devices because they affect the control network.
However, they are unwilling to setup a seperate network so the traffic will be unaffected/at risk.

You really can't let conveniance control your security policy or network infrastructure. Regardless of what operating system you choose,
and harden. The threat will be there until this is fixed.

Is it a 'perfect' solution? No, absolutely not.
It just raises the cost of attack greatly. Defence in depth. Strong access controls to
this seperate network, and strong intrustion
detection. Network IDS are useally a passive device, that can't cause said dammage. Simply
wire a read only ethernet cable if doubt this.

The cost of this seperate network would be far less than the cost of even a minor outage.

However, far more is needed to really secure the infrastructure. It isn't about "drop in" solutions, its about identifying threats and realistic prevention measures.

#

I agree completely with your assertion that a separate network would cost far less than even a small outage. Unfortunately, a combination of non-technically oriented managers,and/or bean counters, and a level of geek myopia previously undetected in the world,has surfaced.
The fire alarm industry has good examples of secure networks that legally require isolation and fail safe structuring as a part of the design. Major grid outages, I will assert, are just as critical, if not more in many cases.

#

I read the article until I got the word virii. There's no such word.

#

1 viri
2 virii
3 viriii
4 viriv
5 virv
6 virvi
7 virvii
8 virviii
9 virvix
10 virx<nobr> <wbr></nobr>...
50 virl<nobr> <wbr></nobr>...
100 virc<nobr> <wbr></nobr>...
500 vird<nobr> <wbr></nobr>...
1000 virm

#

I read all already posted articles and I haven't seen one pointing to the real cause here: at the USA you are proud of your ultraliberal economical system. Well, it can't be so bad, since USA is the more powerful country in the world, *but* is doesn't mean that everything's OK with capitalism when you take it to the extreme.

The blackout is due to the fact that your economic system presses companies to maximize benefits. Some times this -due to the hidden hand, ends up being a benefit for all the population, some others, economical pressures take companies out of the "social path". You just can't maximize benefits (as all companies are asked to by the system) while dedicating expenditures to anything that is not directly in line with the major incoming division of the company.

Even more, companies are pressed by the system not only to maximize benefits but to maximize them on the short run. Then, security and refurbishing is the tipical issue to be "forgoten" or left for tomorrow (after all, it is still working and nothing bad happens, isn't it?).

You know the answer: for some socially critical bussiness (national security, health care, transport, energy...) you have to be ready to accept them to be under public hands (you can try to give to private hands, but regulations and inspections might *need* to be so draconian that no private company is able to make benefits from it).

#

I can only assume that you are not American, and that you are basing your rant on a stereotype of America.

Contrary to your assumption, the American power industry is a government-regulated monopoly. It is probably the most government-controlled industry in America.

Therefore, you can't use this incident to prove your socialist economic theory.

I could continue with a discussion of whether capitalism or socialism is better at providing various goods and services, but I don't have the time, and this isn't the place to do it.

#

Couple of comments.
Background

1. I have 13+ years in the automation/process control industry, 6 years in municipal, and 7 years in the oil and gas industry. I have an additional 5 years experience in mobile dispatching. I was the senior architect of an application used to manage the instrument data sheets for large petrochemical facilities (safety critical information for end control devices). The software that I designed was installed at companies such as BP, Amoco, Saudi Aramco, Hoechst Celanese, and Koch.
2. Windows APIs dominate the control systems industry today - OPC (Objects for Process Control) is a dominant API. There are still many old systems that run on UNIX, but most new systems are Microsoft based.
3. There are many Windows based control packages (Wonderware, Intellution Fix, etc.). During the tim that I worked for a controls consultant in the Oil and Gas Industry, the majority of the control systems installed were windows based. The percentage of windows based control systems increased over time.
4. AFAIK, Microsoft is not in any way shape or form liable, if a defect in their software, intended or otherwise, causes a catastrophic control system failure, resulting in serious economic loss, injury, or death.
5. Operating systems used for primary control should be pre-emptive, and deterministic. Microsoft does not make a pre-emptive, deterministic operating system, yet is dominant in the control systems industry. Most windows based control systems run windows on top of a real time executive, that makes windows more deterministic, for real time applications.

Comment

I know that at present (as I speak), a major gas utility in the US, one the largest gas utility companies in the US, has very severe networking problems in their system, caused by viruses, and the software used to detect and fix those viruses. The virus problems have totally halted a major critical IT project on their network. The engineers that I have spoken to believe that there is a high probability that their system will fail completely because of viruses, caused by Microsoft Operating systems running on their network.

Such a failure could halt the distribution of natural gas within the US. One can only speculate as to the effect such an event would have on the economy, on the stock market, and on the world economy.

IMHO, the design flaws and security vulnerabilities of Microsoft technology constitute gross negligence and irresponsibility. I further believe that there is a very high probability that Microsoft software will cause a serious and catastrophic control systems failure, resulting in major economic disruption, serious injury, or loss of life.


 

#

I agree. It is all down to the attitude of those in authority to what is providing or benefit, and what is simply making money for its own sake. Here in the UK we had the most reliable and well-engineered grid in the world. Being a small country with quite high population density, that was probably quite easy to achieve, over 100 years or so of cautious development. We laughed at the last big blackout in the US many years ago, because that kind of thing could not, at the time, happen here. (That one was a hardware problem, with incorrectly coordinated circuit protection, something we have traditionally handled in a rigorous manner.) There were a very limited number of generating authorities, AFAIK one for all of England and Wales, one for Southern Scotland, and another which provided by far the cheapest power, the North of Scotland Hydro-Electric Board, (fuel cost = zero, most plant costs written off 50 years ago, net running cost (almost) a few men with oil-cans) who betwen them provided power to the one grid. I used to live in Scotland, where the weather is not necessarily ideal. Power interruptions of over an hour happened only twice in 20 years, both due to lightning, one lasted 3 hours before it was repaired. Most interruptions were seconds or minutes, and even then they were so rare... They were also confined to the local area served by the 11Kv grid, outages of the 132Kv and 400Kv supergrid were almost unknown, and the high voltage network had a vast amount of redundancy to allow quick re-routing. The lower voltage lines would only have redundancy in major areas, although 11Kv lines were usually fed from both ends so a fault meant that someone had to go to a pole in the middle of nowhere with a key, open an isolator, go somewhere else, and close another isolator, to restore power to most consumers. This could take a few hours in rural areas, but in view of the number of people affected, was quite satisfactory.A high voltage fault would have all this done, in seconds, in the control room, because millions of people would be affected. Since Mrs. Thatcher's vile government, the whole lot has been privatised, the far too many generating companies are making economies which are now known to have left the country with insufficient capacity to deal with a cold winter. I live in London now, in the last 5 years we have had several interruptions of many hours, despite the fact that the grid is mainly underground and fairly well protected from lightning. One interruption affected a large part of NW London for about 24 hours, they repaired it, and it happened again.... It was allegedly a 132Kv grid fault (must have been, from the area affecetd), so where was the redundancy, required by law if sufficient consumers are affected? I guess some rotten old cables had failed earlier and not been repaired, so the redundancy had been lost. Another time, Christmas in Staffordshire, there were high winds. Years ago, overhead lines were maintained, but not now. The 11Kv lines had stretched and were so slack that they wrapped round each other in many places, in addition to those that simply broke. No power for over 48 hours that time. It is getting worse..... There are many brownouts, and transient interruptions. One well unified organisation worked best, we now have a mess where I can buy my electricity from any number of suppliers, on different terms, even from the gas company! There are only one set of wires under the road! A beaurocratic nightmare, created to supposedly cause competition, to benefit the consumer. As in all matters of safety or reliability, one well-defined organisation works best, and is the only way it should be done, but now we are on more or less the US model, with far too many generating organisations, and constant near real-time wheeling and dealing on price to the grid (which is no doubt why there is the need for so many computers). The profit motive reigns supreme (and electricity costs MORE to the consumer in most cases...)

The way to get relaible supply is to create a single responsible authority, owned by, and responsible to, the community at large (usually the nation). They can use sub-contractors for maintenance, construction, etc, but they and they alone must make opertaional decisions at all times. This used to be the case, till greed intervened...

My parents used to live near a power station which was built around 1959, and was controlled by analogue technology. Yet, around 1962, they were able to fully automatically start up from cold, synchronise and take their share of the load on the grid. This has now gone. The new power station 2 miles away, also a coal burner, is bigger, better, I daresay more efficient, and stuffed with 1970's PDP-11 or similar technology, quite reliable, not in need of internet. BUT (and a very big BUT) because they can produce electicity at a variable cost depending on how much is called for, how many turbines they have running, the cost of coal and lots of other factors, and the "grid" (still run by one organisation, but entirely separate from those who run the generators) is willing to pay a variable price according to even more variables such as the prevailing wind (literally, some base load is handled by wind power, which needs to be kept running when possible to justify the high capital cost), there are now systems in place which are doing near real-time demand forecasting and "horse trading".As many of these parts of the system need to be accessible to accountants, who understand spreadsheets (just about!), you can guess whose OS then gets involved, and of course this is sending demands to the control room staff or directly to their previously well-behaved control systems. It will all end badly, as in the US. I am seriously expecting major problems this winter, and will be making my own backup arrangements.

Ironically, it was capitalism that created the grid system in the UK, and a different flavour of capitalism that has started its destruction, same with railways, telecomms, postal services......
Politics is generally destructive, only proper engineers and managers should be running these things, and any profit they make should be re-invested to improve the system. Things like these can only run as integrated systems, not an odd collection of adversarial organisations.

In the case of electricity supply, the other alternative is to divide it up and have no grid, only local generating companies. It is actually more efficient, but the redundancy is lost. I would suggest a third way, to have a grid in place, but with breakers and isolators normally open so it runs in entirely separate sections, with provision to maintain synchronism, then cross-feed from one area to another may be selectively enabled if a power plant is going down for maintanance etc. In the event of a fault, cross-feed would not be enabled until the failed area had been cut up into smaller pieces by section isolators etc, these would then be closed one by one, with cross-feed from adjoining areas, so that there was no possibility of dumping a massive fault on the whole grid. But, to work, it would need a bit more generating capacity to be kept operational and certainly more wire and switchgear, which the accountants would not like.....

Interestingly, on multi-engine aircraft, the power systems usually operate that way, with a DC bus fed from each generator, one on each engine, and a battery also on each bus. There is a cross-feed usually known as the Bus Tie, which is via a very high integrity circuit breaker, and is only closed by the crew, and then after careful consideration, if there is a gross failure on one system. When the Bus Tie is open, there is absolutely no possibility of losing the lot due to a single failure, with it closed you get power available everywhere but a gross fault will then take out the lot, disaster probably following... Closure of the Bus Tie generally illuminates a very obvious warning in the cockpit, as a reminder that the power systems are at greater than normal risk of total failure.

I think that any grid sectioning system will cost money, but you have to spend (invest) money to create or improve things. Maybe accountants and politicians can't understand that.

#

There's a great old commercial, where the young teenage-cracker stereotype trys to break in and steal the Cadbury secret. Cut to a smooth-running electrical grid cum 'Chocolate Security' control room. Everybody is synthesizing the data and pinpointing the exact cause of the problem.

Close up of the steely-eyed commander getting all this information and saying "Shut 'im down!". The teenager room and the whole block go dark. The commander does this knowing full well that there is a hospital next door, with a surgeon about to slice into an open heart, and a high-rise with a pregnant lady in the elevator... But the fate of humanity is at stake...

Now the modern verson -- Cut to the control room with all the little guys yelling "The computer's locked up! Third time this week!" "We don't know what's going on!". Zoom in to the commander who slowly rolls his eyes and dies of a heart attack.<nobr> <wbr></nobr>..And the whole world goes dark...

#

. . . it contains the word "virii" for "viruses". Uninformed nitwits who think that this "word" shows off their education are often the same ones who think that you can cause a machine to fall over by waving a copy of Windoze at it. See http://www.perl.com/language/misc/virus.html for more discussion of the "virii" inanity.

#

"Is it in the realm of possibility? OK. [Is it] in the realm of probability? That's another case."

Yeah, it would be like hijacking four planes simultaneously, and getting three of the four to hit their targets within minutes of each other... pretty improbable...

#