Growing business: Linux clustered solutions

It’s the ’80s all over again. We’re recovering from another
energy crisis. There’s a conservative Republican in the White House who
wants to build a Star Wars missile defense system . And in the IT
sector, supercomputers are hotter than have been for nearly 20 years.
Fortunately for the Linux community, the newest and best systems
being proposed and funded are going to run on Open Source software.Now more than ever, there are big markets for supercomputers.
Financial services markets, oil exploration companies, biotechnology developers
and a variety of defense and security applications need the type of high-powered,
multi-threaded calculating power offered by supercomputing architectures.
But not everyone wants to pay the big bucks associated with proprietary
supercomputers. The systems are incredibly expensive, but the biggest
cost has been developing applications and attempting to port those
applications from one generation of system to the next as systems grew more
sophisticated. Beginning in the mid-’90s, IT pros in a variety of academic and
research organizations found they could create their own “chop
shop supercomputers” by clustering cheap or discarded machines
using Open Source technology. Today, institutions and companies who want to
roll their own can get a copy of Open Source code such as Cplant, a cluster
codebase developed by the U.S. Department of Energy and available as a
download from the the government’s
Sandia Labs.

But while code may be available, it isn’t necessarily easy to use.
Over the last year or so, it has turned out that developing the software
and expertise needed to construct these clustered systems is a
business — a real business, and one that promises to re-ignite some of the early
enthusiasm of the first wave of Linux investors.

Linux-based clustering systems are starting to spring up all over
the place. Take the case of Eagan,
Minn., a well populated suburb in the Minneapolis-St. Paul area. While the Twin
Cities area has had its share of high-tech businesses, when the locals
talk about of state-of-the-art equipment in Eagan, they’re probably
discussing what’s on sale at Hardware
Hank, where the the most high-tech items for sale may be the lawn
mowers. Nevertheless, not very far from Hank’s, a small group of former
Cray workers have begun a Linux-based clustering firm called Unlimited
Scale Inc. The company’s name predicts just how large co-founder
Steve Oberlin thinks large Linux clusters can grow, if they’re built right.

Oberlin has been obsessed by the idea of building them right for
better than five years. Back then, Oberlin was the v.p. of hardware engineering
at Cray Research. Oberlin had long been interested in using a clustering
approach to build the systems needed to address complex phenomena such as weather research. He spoke regularly at places such as the National Center for Atmospheric Research, a consortium of universities pooling resources to better understand weather-related phenomena.

Soon Cray was absorbed and partially digested by Silicon Graphics.
Three years later, Oberlin found himself heading up Cray/SGI’s software
program. Oberlin noticed the difficulties that his customers had when
porting their complex systems from one machine (and operating platform) to
another. He suggested that among the biggest barriers developing and selling
ever more complex systems was
the lack of operating system transparency or the ability to modify
code cooperatively along with his clients, all making code migration
difficult for supercomputer users. Before he was in the job six months, he was
already suggesting to listeners that Linux would make more sense as a operating
platform for such clients. By October 1999, SGI was trying to spin off
Cray with Oberlin as head of the business unit. But the planned sale to
the high-tech turnaround shop Gores
Technology Group began to sour almost immediately. Gore, best known for
its acquisition of The Learning Company, kept trying to lower the price. By
March 2000, Cray was bought by Tera Computing, and Oberlin was headed
back to his garage in Chippewa Falls, Wis.

Over the last year or so, Oberlin has collected Cray alumni from the
greater Twin Cities area to build a new kind of Linux-based
cluster — one that he thinks can scale almost infinitely.

Today’s Beowulf -type clustering systems are essentially cobbled
together using Ethernet — a process that makes them relatively easy to build but
which quickly drains resources as each new unit is added to the
cluster. Past a certain size, individual nodes begin to lose a lot of the
efficiency that made them attractive in the first place. According to Oberlin,
larger clusters spend far too much time with internal administrative duties,
not nearly enough on computing. So he has configured his Enter Oberlin.
Unlimited’s solution involves rejiggering Linux to cluster nodes together to run
dedicated tasks such as managing peripherals while letting others manage the
straight ahead processing.

Oberlin has received financial assistance and management help from
local entrepreneurs. Chief among them have been the four principals of the Quatris Fund, a small ($15 million) satellite fund of St. Paul Venture Capital. The fund is managed by four former CEOs from the area. They include Minnesota radio station owner Steve Goldstein, Gary Smaby, a Piper Jaffray analyst who had tracked the supercomputer industry during its heyday, electronics entrepreneur Erwin Kelen, and John Rollwagen, the former CEO of Cray who was widely credited for the company’s
successes during the ’80s.

Although part of a “seed” funder, the principals consider don’t consider
themselves to be speculators. They expect to have product (and customers) before
the end of the year.

Unlimited is joining a rapidly growing crowd of Linux-centric
clustering companies. Clustering specialist Myricom continues to grow rapidly
while being profitable since 1997. Firms like Salt Lake City’s Linux NetWorX
are creating product and attracting financing. Linux NetworX just
raised $5 million. Others are coming rapidly.

Not all clustering technology is being used to build supercomputers
or forecast the weather. And clustering technology isn’t necessarily
limited to Linux, or developed by Open Source religionists; Berkeley,
California’s PolyServe Corp. is developing
clustering software and storage management products based on Linux and Solaris.
The company has attracted financing from Greylock Partners among others. Mountain View, California’s SteelEye
Technology is using clustering technology to provide redundancy and
improve the performance of mundane applications such as e-commerce transactions
and Internet data centers. While the company supports SuSE, Red Hat,
TurboLinux and Caldera distributions, it has also ported its application to
NT, Solaris and Win2000 platforms. The company’s clustering technology
and operating system agnosticism has won the company over $30 million in financing
from the likes of the Intel 64 Fund and Venrock
Associates.

But whether the application is to provide a fail-safe e-commerce
server, or a supercomputer capable of knocking down an incoming missile,
clustered systems running on a Linux platform seem to be the design of the
future.