Linux grids take over high performance computing at Penn State

Linux-based computing grids are becoming the default setup at Penn State University’sHigh Performance Computing department, thanks in part to the Emerging Technologies (ET) group at the college.The group investigates and reports on IT technologies, architectures and strategies that it believes would be beneficial to the school. With a desire to do scientific research better than it ever has, three years ago Penn State began implementing Linux clusters. “There’s a huge commitment to high performance computing,” says Al Williams, the team leader for ET.

Some of the things PSU is doing with high performance computing include materials research, computational fluid dynamics, computational chemistry, genomics, weather and climate modeling — the type of calculations the National Science Foundation (NSF) calls the “grand challenges of computer science.”

Back in its mainframe days, PSU used a lot of AIX and Unix, but lately Williams and his crew have made advances using Red Hat Linux on Dell, Sun, and Angstrom hardware. There are still some big IBM/AIX guns in use, but Williams likes the Linux clusters. “We have had great success running large arrays, 170 nodes, 340 processors running Linux.”

One of the challenges ET faces is maintaining balance in its growth. “You have to decide where to put your money,” ET group leader Jim Leous says. The problem is to ensure that there is enough storage, enough memory to keep the network running smoothly, enough processing power to perform complex calculations — and the list goes on. Leous says the focus has been to try to balance everything better and not let one aspect get too far ahead of the others.

Williams agrees, saying the ultimate goal is to “get the most science” out of the clusters. “We’re not looking to beat all the benchmarks, yet not be able to do real research.” For example, the group chooses not to connect nodes using commodity Ethernet connections. “We trade off getting the maximum bandwidth versus how fast it takes to initiate communication between the machines,” Williams says.

One way Williams and the ET group have been able to economically expand the computing power at PSU is by using the “condominium” approach. If one department only has enough in its budget to buy 70 Xeon servers, but another department could buy 100, the two departments can pool their servers and take advantage of the power of 170 nodes without overspending.

Leous says he’s found they can provide thirty to fifty percent more computing power on average because there’s already an infrastructure in place because “there are economies of scale in building these clusters — by pooling the money, each of them can have access to much greater power than they could alone.”

An upcoming project for ET is more practical: server consolidation. “IBM is providing Linux on the mainframe with special processors that enhance the performance of Linux — we’re implementing that in our administration information systems,” Williams says. In what is slated to be a two-year project, ET has purchased two zSeries servers and is negotiating with Novell for its SUSE Enterprise Server software. Workstation applications will not change, at least from the users’ point of view. “Access to resources will be through a web interface,” Williams says. “In the end it will not matter what the desktop is.”