Predicting the weather with Linux at the National Weather Service

105
– By
Norbert Cartagena

Government institutions are often maligned for their spending habits, especially when it comes to computer hardware and software. With the increasing popularity and reliability of open source software, more and more government institutions are looking to Linux to help reduce their computing costs. But can Linux really live up to all the hype? The National Weather Service (NWS) thinks so. In fact, the implementation of Linux based solutions within the Service has not only helped decrease costs, but has also helped increase productivity.

Paul Kirkwood, Branch Chief of the Dissemination Enhancement Team at Southern Region headquarters, says, “I do see the future of the weather service moving towards the Linux environment.”

Kirkwood was a part of the team that, in early 2001, brought Linux to the attention of the NWS. They looked at staying with their current HP-UX servers, but also studied the Linux alternative. “This was a group effort,” he explains. “We put together a team and asked ‘Where do we need to go in the future of the weather service? Where do we need to be?'” They found that they got a two-to-one performance advantage by moving over to Linux on Intel hardware, according to benchmark tests, and received approval for the transition from both the CIO and the Director of the NWS.

The march towards Open Source

The Service expects to convert their offices and research centers to Linux over the next three to seven years, possibly as soon as the end of 2005, depending on their budget. This includes over 2,000 systems in approximately 140 geographically dispersed locations across the United States and its territories, including all 122 forecasting offices, 13 river forecasting centers and 4 National Centers for Environmental Prediction (NCEP). Each will have a high-availability system with design availability of 99.9%, including full redundancy and a high-availability clustering scheme on the existing servers.

Linux first appeared on the NWS scene in 1995 in the form of Slackware, which they used for their quantitative precipitation forecasting (QPF). Today, they use it operationally for a number of their servers, and are in the process of migrating from a Microsoft Windows-based primary domain controller (PDC) environment to a Linux-based PDC. They also use it extensively for their Advanced Weather Interactive Processing System (AWIPS), allowing them to prepare their forecasts and view their model data. The increase in their Linux usage has been due largely to its advanced security features, portability between different platforms, and the availability of software packages.

According to Barry West, Chief Information Officer (CIO) of the NWS, cost also has something to do with it. “They’re significant savings,” says West. “Up to a 3-to-1 savings ratio.”

AWIPS, which currently consists of five workstations and four servers, plus one external server, is the Service’s operational forecaster’s system. It’s a tool for meteorologists and hydrologists, which allows forecasters to visually integrate meteorological information they need from surface observation, satellite data, and Doppler RADAR information in order to generate weather warnings and forecasts on one three-headed desktop. It currently runs on a number of J-class and K-class Hewlett-Packard servers, as well as J-class workstations running HP’s proprietary HP-UX operating environment. The NWS, however, is in the process of migrating from their HP-UX servers and workstations to Intel-based servers and workstations running Red Hat’s Advanced Server, making this their baseline standard. According to John Potts, Senior Budget Analyst, they’ve completed the first phase of the AWIPS transition at the cost of $8 million, with another $2 million being spent this year on the project. This move saves them on both hardware and maintenance contract costs, which, Kirkwood explains, “[get] rather high.”

Another reason for the switch was their desire to get on a three-year periodic replacement program cycles. According to Chuck Piercy, AWIPS Program Manager, due to the high availability of Intel compatible parts, as well as both the competitive pressure on the hardware providers and the performance bang for the buck, this isn’t a problem with Intel-compatible components. “Price-wise, [the move to Intel] is very nice,” says Kirkwood. “We can plan for expandability in the future… and have the hardware ready to go, running Linux, knowing that it will support just about anything we do.” Conversely, because of the large costs incurred, this wasn’t possible with the HP-UX systems, which required an entire system replacement, and could only be done approximately once every decade, continually putting them behind where they needed to be technology-wise.

Linux in action

So how does Linux compare in performance to current NWS systems? According to Kirkwood, their current HP-UX systems can take up to eight minutes of start-up time to load their software should the system go down, while Linux on the Intel platform takes under a minute. In addition, the switch has helped increase data processing speed.

“The performance that we measure with our benchmark has increased by over 100% since we completed phase one of our Linux migration,” happily boasts Piercy.

Nearing the end of the first phase of transition, they’re already using Linux in a number of critical systems, including two eight-processor Dell units with eight gigabytes of RAM, each with redundant backup, and each loaded with Red Hat Advanced Server 7.2 running kernel 2.4.20. They use these to handle more than one million hits a day. In fact, as Kirkwood points out with pride, “with the last two hurricanes that occurred this fall, we served petabytes of data with these servers.” They’ve also brought in ten Penguin Computing Relion 1U rack-mount Servers, with dual Intel Xeon CPUs and four gigs of memory, all loaded with Apache, to replace their current Microsoft IIS servers.

As for their AWIPS systems, according to Piercy, they’ve brought in over 600 Dell PowerEdge 1650s and 2550s (about 300 of each), along with 30 Dell Dimension 420s, and 280 Intel Xeon-based IBM IntelliStation M Pro workstations. In addition, they’ve added two Linux workstations and a Linux High Availability Cluster using Red Hat Advanced Server. They’ve also ported an older communications processor, which supports their satellite broadcast network (SBN).

Due to some of the software used, they currently use a number of HP workstations with three monitors. On these they expect significant hardware savings once they migrate to Linux. According to Kirkwood, whose computer is graced by an nVidia 64 megabyte dual monitor card and a 32 megabyte nVidia PCI card, by using what he refers to as “gaming cards,” they’re able to take some of the load off the systems with their graphical models, including better refresh rates and alleviating some of the swapping between the hard disk and the processors. Both GNOME and KDE have the capability to recognize the three screens, allowing them to have one big screen or desktops one, two, and three, which will make their systems more meteorologist friendly.

Aside from the coolness factor that comes from having such powerful systems running Linux at your work place, a pride Kirkwood is admittedly guilty of, one of the bonuses has been the fact that for all intents and purposes they’ve been able to “drop and run” with Linux. The software they use is mostly software included on the Red Hat 7.2 CDs. For the desktop, they’ve been using OpenOffice.org‘s suite in lieu of Microsoft Office, and if they must boot up Windows, they do it with VMWare. In addition, one of Kirkwood’s favorite advantages is the availability of numerous Linux user groups, both at home and abroad. Support from the community “is excellent,” he says. “To me, that’s a plus. A huge plus.” (That community support, I’m happy to report, includes articles and information found in Linux.com).

It’s not all blue skies

Although in the back end things are going all Linux, the desktop is another matter. There haven’t been any major problems, but not everybody is up for the change, even with tools like OpenOffice. “What we found,” explains Kirkwood, “is that as Linux and Open Source have progressed, [we’re] getting to the point where the Linux desktop can do almost anything that the Windows desktop can do… There’s talk of going to the Linux desktop. There’s also talk of going to Microsoft and .NET… The [Windows] desktop will be the hardest to convert people off of. It’s hard convincing people why they need to switch what they’re used to.” He points out, however, that as for the people using CDE, the change will be welcome. “Since much of [the meteorologists’] work was just clicking on an icon and making this work, this was simply a matter of teaching them how to click a button. For these systems, the main question is ‘How do I view the data?’ Provided that the system has a way of doing it similar to their old way of doing it, they have absolutely no problem. This is where GNOME [came] in handy… They saw no difference except when it came to performance. Then they actually got to see the difference.”

In contrast, the hardest part, according to Piercy, is getting their antiquated Informix database server replaced. In addition, they’re still using a messaging system based on x400, a precursor to modern-day LDAP. They’re looking to do this with TCP/IP and SMTP, but the high availability requirements might require modifications. This, along with replacing the three-headed HP workstations, is part of the second phase of Linux integration expected to begin at the end of 2003.

All things said and done

With the use of Linux, the NWS has seen decreases in cost even as they increase their computing performance, which, through systems like AWIPS and Mesoscale model forecasting, allows them to get more accurate warnings quicker in order to protect the public. And fast, accurate weather forecasting is what the National Weather Service is all about.