November 2, 2001

Running OmniCluster's Slotserver cards through the wringer

Author: JT Smith

- by Brian Aker -

At the most
recent LinuxWorld
, I
happened upon a little booth with some guys selling computers on a card,
something I
have seen before but that is still rare enough to catch my attention. Especially
cool
about these cards, though, is that they're PCI cards acting as loopback devices
to
the host computer, controllable by the host.

Loopback you say? The cards communicated via
a logical ethernet device that only the host could see. This means sockets, and
this means you have bus-speed ethernet. This results in a very simple socket
interface to program against.

The OmniCluster SlotServers 1000 are half-size PCI
cards with an internal IDE adapter, one
SODIMM slot (my board had 256 megs onboard, but I'm told they are testing 512
meg
SODIMMS) and an internal plug which allows you to attach an adapter that will
let
you use an ADB keyboard. On the card's edge is a 10/100 ethernet port, a USB
port, sound ports, and a VGA output plug. There is also a male PCI card
adapter on top of the card that will require an adapter to use (and a pretty
funky
case to support). I was told the male PCI card is for testing, but waste not,
want not. By attaching a male-to-male adapter to it, it will take a 3v PCI card
(even though you will need a fairly odd case to support something like this).
So it's pretty much a standard PC on a card.

The SlotServers I received had an additional goodie as well, a 20-gig IBM IDE
drive (they sell this as an add on for $299, you don't need it though since
the card can use any internal IDE drive and can even share a drive under
the right setup). This turns the half-sized cards into full-sized cards, but in my case
brought
with it the complication that the cards didn't quite match up with the
full-sized
card slots in my case. Because mine is a tower case, the cards sagged a little
(they
still ended up being supported, but not quite exactly right). I also had the
nifty
idea of sticking one of these cards in my G4 Mac, but the full length cards, it
turns
out, won't fit in one with the add-on IDE extension. Taking off the extension
and hooking it to a different drive in the case worked fine. Doing something
like
this with a non-supported host will still leave you with a card that works,
because
all it really needs is the power from the PCI bus, but it won't be controllable
by
the host.

The cards I received had their host drives installed with a standard Red Hat
7.1 distribution. They behaved just like a normal PC running Red Hat. The cards
can
run
Apache and will work as a DHCP server, and because they are sitting on the bus
and have an
ethernet port, they make great firewalls for local networks. Each card is
a Linux
box on a
card with an obviously small footprint (effectively zero footprint, if it goes
into
a box already on the network), so you can do everything you normally expect to
do with a Linux box.

Now let's take a different sort of problem: Anyone who has worked with the Web
has
run into the problem of separating Web servers from the servlet
engines that produce dynamic content. For a first round of testing, I downloaded
IBM's Java SDK and installed it on an Omnicluster card. I then set up the Java
servlet engine to run locally on the card. I compiled mod_jserv on the host
computer and pointed it to use the servlet engine. Presto! I had a host computer
using an Omnicluster card as a dedicated CPU for running Java applications,
while letting the
host handle the rest of the work.

Then I found that, because the host computer had more CPU/memory
than the OmniCluster cards (the host computer is a dual-processor Intel box with
2
gigs of memory,) it had more cycles to spare than the OmniCluster card. So, I
reversed the setup, using the host as the CPU and letting the Omni card handle
the
incoming traffic. The database I was using was also on the host, and this
allowed faster connections to the servlets, and the combination worked quite
well.
So now I have a setup with the host computer running servlets that are talking
to
the DB, and the OmniCluster card running Apache.

I was not completely happy with this setup, though, because if I were to lose
the
servlet engine I would have a problem. I'd have to wait for it to come back
up before I could begin to serve more Java pages. The solution for this? I added
an
additional OmniCluster card and placed the servlet engine on it. So you now have
one
card serving pages, with another card handling servlet requests, and finally, a
host
running the database. All of the communication happens over the local bus, which
keeps the need for security between databases/servlets/webservers to a minimum.
Finally, the host computer can reboot an Omni card if it needs to, so I have a
simple
way of rebooting the cards if they should lock up, although I should mention
that
this never happened during my tests.

Pretty slick -- and it was all Linux, so it was trivial to do. I am told that
the
cards will also run Windows so those of you who need rebootable Windows boxes
might
find something of interest with this.

I also toyed around with a similar setup using mod_perl, letting the OmniCluster
cards work as Webservers and the host as a mod_perl proxy engine. (The setup for
this sort of environment is documented in the mod_perl FAQ). I've frequently
consulted for corporate clients seeking secure networks for their e-commerce
sites who
needed to use multiple hosts. Because rackspace was an issue in most of these
situations,
these cards would have come in handy.

The database I had been using on the host for testing was MySQL. One of the
problems
we have with MySQL on Slashdot is that when we are generating a lot of dynamic
content for the site, replication can get backed up. (We use replication to keep
a
current version of the site and to populate a database that is just used for
searches). So I've been looking for a way to ensure that replication always
stays up
to date. I tried a local test to see if something like the OmniCluster cards
would
work well for this. I was able to download the RPM from the MySQL site and
install
it on the OmniCluster cards. Via NFS, I set up MySQL so the OmniCluster cards
could see the same database files as the host. I then enabled file locking for
MySQL, and was
able to select off the same tables as the host. While this was an interesting
hack,
MySQL needs a better distributed lock manager than system file locks, so this is
probably not a very practical use. (Now, if the OmniCluster cards had
fiberchannel
or SCSI it would be a very different story. Then you could share
drives directly).

The only question I really had left in my mind after all of this was the
cards' binary
compatibility. For a real test, I hooked up a CD-ROM to the OmniCluster
card's IDE bus and inserted my fully functional copy of DB2. After going through
the install, I fired up DB2, added and dropped some tables, and ran a few Perl
scripts
to test it. Worked like a charm. The memory on the cards (just 256MB,
remember), did not make running DB2 a very practical case study, though.

I have to say, even though rack space is not an issue in my office, the
Omnicluster
cards are nice, because I can keep them in my desktop computer, use them for
other
development work, and never worry about trashing my "main" box. I don't like
adding boxes in my office for noise and heat reasons. Because the cards do not
have fans on them, the general decibel level of my office was not changed.

The only real problem I found with the cards (other than the fact that they
don't
line up well for full-length cards) was that the software to control the cards'
install was a little clumsy. I was even able to initially crash one of the cards
by
not following the instructions completely. (There was also a problem with the
IBM add-on hard
drives failing, but I blame that more on the original packaging than any failure
of
the cards.) I do wonder about re-installing the cards, though, if you are using
the
IDE card extensions. Unless you keep a bunch of spare CD-ROM drives in the host
computer for the cards to use, you'll need to open up the host to attach a CD
drive.
There are a number of ways around this, but still it is an issue you should keep
in
mind. I am told that in the near future you will be able to use local disk
images from the host to
run the cards, which would also make them a lot easier to manage. That would
probably make for a nice solution for the next up-and-coming Rackspace.com
look-alike that wants to sell dedicated servers, but has issues with dedicating
an
entire full sized PC to each client.

The list price for a SlotServer 1000 was $499 with 32 MB Memory, $549 with 128 MB,
and $619 with 256 MB. Shop around, though, because these are list prices and at Linux World
they were going for quite a bit less.

In the end, if you are looking for a solution that requires secured networks
between
hosts, want better use of space or just need a contained development
environment, you
should give the OmniCluster cards some consideration. Plus, if you consider how expensive four-way and eight-way processor boxes are, and are not
scared of
a little distributed processing, these cards make a great way to solve problems
for traditional client-server computing.

Brian Aker is senior programmer for Slashdot.

Category:

  • Unix
Click Here!