Author: Jay Lyman
Weather is a tricky thing to predict, but advances in Linux-based High Performance Computing (HPC) are helping climatologists and global weather modelers achieve faster, more accurate results.
Thomas Zacharia, associate lab director at Oak Ridge National Laboratory (ORNL), said the Linux movement has had a strong impact on weather and climate research, as well as other scientific areas requiring vast simulations,”because obviously you’re harnessing the collective knowledge of large numbers of people around the world.”
Zacharia said ORNL’s 456-processor Altix system from SGI running Linux delivers six times better performance than the previous generation of systems from a variety of vendors. Now, Zacharia said, most of the major vendors have delivered successful, high-power computing solutions with Linux to model and study the weather — SGI’s Altix, Cray’s Red Storm and IBM’s BlueGene among the most prominent. Those systems, he explained, are giving researchers better results than they got from the systems they used to use.
“The advantage of Linux is, operating system plays an important role as
you build systems of this scale,” Zacharia said. “The ability to efficiently
scale the operating system along with the machine is crucial.”
Referring to the humongous simulations that are used in climate
research, Zacharia said scientists are benefiting from a good balance of
increased processing power and memory, better interconnect technology, a
sturdier OS — Linux — and advances in mathematical algorithms.
“Without a doubt, we’re in a wonderful era with a number of purposeful
systems and machines that are entirely focused on the scientific and
technical market,” he said. “I believe computing is going to be the great
enabler in science and research in climate because there’s no other
way.”
Usual benefits of Linux
That focus has proven fruitful for SGI, according to
manager of environment marketing Jill Matzke, who said HPC users in general
are shifting from proprietary systems based on RISC processors and Unix variants
to more industry standard-based systems with the main drivers being
price-performance and system portability. SGI has used its Altix system to
help fuel the transition for the weather and climate community as well as
the broader HPC market, according to Matzke.
“Researchers are gaining cost savings and interoperability from volume
market components like Linux and Itanium 2 while still enjoying the powerful
performance of the SGI Altix,” Matzke said. “They benefit from the
incredible resources of Intel in delivering top-performing 64-bit
processors, which combined with Altix architecture offers superior scaling
of the very large and complex models that weather and climate researchers
must use.”
Matzke explained that with terabytes of data and extremely complex models
to deal with, researchers also have trimmed HPC budgets and need to cover
all of their needs — computing, data storage, and visualization — with
those limited funds. SGI offers the compute piece with its servers running
Linux or a complete end-to-end data management solution on Linux as well,
Matzke said.
“By offering all of these system elements on Linux, SGI leverages open
source technology on behalf of our customers in weather and we save them
money without any compromise on performance,” Matzke said.
SGI is looking to extend its Linux weather forecast by providing data
management solutions along with climate research compute power, and soon the
Mountain View, California-based company will offer a visualization solution
as well, Matzke said.
“The challenge is getting all the necessary system components to work together on Linux, and this is SGI’s focus,” she said.
Modeling marches on
To tackle some of the toughest problems in HPC today — climate, as well
as life sciences, biotech and nanotech — researchers need systems that can
scale to the data and degree for running “so many years of climate in a
day,” according to ORNL’s Zacharia.
Matzke said because weather and climate scientists are looking for as
much compute power any vendor or budget can give them, they look to systems
such as the Altix to deliver “truly productive compute cycles that scale
real-world problems, not just industry benchmarks.”
“This makes it possible for them to complete model runs in minutes or
hours where it used to take days and even weeks,” Matzke said.”And when it
comes to production weather forecasting, when time-to-solution is paramount
in making timely forecasts, the power of these systems allows forecasters to
run higher resolution and more complex models in the alloted time, thus
increasing forecast accuracy.”
John Parks, deputy division chief of the Advanced Supercomputing Facility
at NASA’s Ames
Research Center, said there is indeed a migration to Linux and other
technologies, including Cray’s X1, in climate and weather research. Parks
said researchers anticipate Cray will release a Linux version of the X1,
formerly known as SV2.
“So even there, you may end up with a Linux platform,” Parks said.
Proprietary clouds clearing
Parks said the main advantage of Linux over proprietary systems is its ability to run on many
different hardware platforms. “However, in the past several years, we have built applications that take
advantage of proprietary operating systems, so when you move (to Linux), you have to (port applications),” Parks said. “For the most part, that’s a trade off that
you’re willing to make.”
Parks said some scaling issues need to be addressed, and that while Linux
delivers flexibility, it may not take advantage of hardware nuances that
made proprietary solutions better in some circumstances. Again, though,
Parks said computer scientists working with climate researchers and the
Linux community manage to make Linux superior in enough other ways to make the open source
operating system a good choice.
Parks said input/output is another issue with Linux “that seems to
be getting rapidly addressed,” and indicated that the challenge now is for Linux to
keep pace with other advances in HPC, particularly new processors from
AMD and Intel.
“Linux has much to do with where the technology is going in general,”
Parks said. “If we can have an operating system available that can be
readily architected for different hardware, it’s better for us, and the
speed at which these systems are capable of operating will allow us to scale
and get (even more) speed.”
Future forecast
Domagoj Podnar, assistant research computer scientist at the
environmental research nonprofit Desert
Research Institute (DRI), said although his organization’s use of Linux came
with the use of SGI’s Altix, he has long seen the advantage of Linux in
clusters to run atmospheric modeling.
DRI, which runs a 13-node, dual-Xeon cluster to run atmospheric models in
real time, assists firefighters responding to wildfires and produces forecasts for
prescribed agricultural burns. Podnar, who gained familiarity with Linux on
workstations, said there is a big push in the industry to run climate models
on Linux PC-type computers “because of its cost advantage.”
Podnar said a look at the most powerful HPC systems in the world and the
prevalence of Linux highlights the operating system’s suitability for
high-performance computing such as climate and weather research.
“I think it’s the way of the future,” Podnar said.
Category:
- Linux
