March 31, 2004

Cool tools for remote administration

Author: Brian Jones

You have to admit that the job of a system administrator would be made worlds
harder if we had to actually sit down in front of each machine we're
responsible for. Why bother having a network if you can't use it to make your
life easier? Let's have a look at a couple of cool remote administration tools
that are both useful and easy to use.

Xnest

Somewhere, there's a 20-year Unix administration veteran groaning that I've
even mentioned Xnest. Why? Because Xnest is a way to query a remote machine for
an actual login screen, and get a full desktop session on the machine in a neat
little window. The Xnest program may be downloadable as a separate utility in
some distributions. In many, though, Xnest is installed along with XFree86, or
installed by default as the XFree86-Xnest package.

Xnest is, as indicated by its name, a nested X server. Technically, it is also
a client. It can connect to a display on a remote server (making it a client),
and respond to input from client applications, like a server. Since Xnest is
part of the XFree86 distribution, Xnest runs anywhere XFree86 runs. As a test,
I installed the XFree86 distribution for Mac OS X, fired up a terminal window,
and typed the same thing I've been typing for some time on my Linux machines: Xnest -query myserver :3

Xnest on Mac OS X -- click to enlarge

This presents me with a login screen, where I can log in to
myserver and get a full desktop session. Note the final argument here,
which tells Xnest which local display to use for the session. Since my local
DISPLAY environment variable is set to :0, I basically set the
DISPLAY variable for the next window to another value. Figure 1 shows a
screenshot from a Mac OS X box, with desktop sessions from Linux and Solaris
machines each inside their own Xnest windows. For what it's worth, Xnest can
also be used to open multiple local X sessions. However, what with the
numerous virtual desktops I have access to through my window manager, I have yet to find a
practical use for this.

Depending on how you configure your X servers, Xnest may work against your
systems right out of the gate. However, the default configuration of many X
servers disables some of the XDMCP settings needed for Xnest to connect to a
remote host. The exact setting to change varies according to the display
manager in use, but generally, the display manager config files are similar
enough that you can open the config file for whichever one you use, search for
"XDMCP," make sure that it's enabled, and restart your display
manager. If it still doesn't work, check the various usual suspects, like
firewall and tcpwrappers settings, as well as your system's /etc/X11/Xaccess
file.

xmeter: The admin's best friend

The xmeter tool is an absolute necessity for any administrator's desktop.
Inspired by an older tool called xload, xmeter basically
queries remote servers for output from the rstatd daemon. Of course, you'll
need to have the rstatd daemon running on any boxes you want to monitor with
xmeter, but getting rstatd installed and running is trivial on most Unix
variants. Once you have that lone requirement in place, you're almost ready to
use xmeter.

While xmeter is a breeze to build from source, configuration is another
story. It's not that configuring xmeter is hard, mind you. It's just that
xmeter is something you might need to tweak over the course of some number of
days or weeks before you get everything just the way you want it. The man page
that comes with xmeter is loaded with seemingly endless options for your
configuration pleasure. Since xmeter doesn't take an option pointing it to a
configuration file, all of the options need to be fed to the xmeter command on
the command line. As the number of options you feed xmeter could run into the
hundreds in a big enough environment, the easiest way I've found to launch
xmeter is through the use of a wrapper file that holds all of my options.

Here's a relatively simple command file that monitors the load on a few
servers in my environment:


#!/bin/bash

/home/jonesy/bin/xmeter -sn -cols 2 -dv -ebd red -ebg black -efg yellow -ehl red
-eibd black -fbd yellow -fbd red -ffg yellow -fhl red -fibd black -h 60 -w 140
-obd black -obg black -ofg green -ohl black -oibd blue -to 3 -update 10 -wbd
yellow -wbg black -wfg yellow -whl yellow -wibd black server1 server2 server3
server 4

I constructed this command by going through the xmeter man page and
picking out the features I wanted in my display. If you take the above wrapper
configuration, change the path to the xmeter binary, replace the "server1
server2..." entries to names of real servers in your environment that run the
rstatd daemon, and execute the resulting bash script, you might very well find
that this config is all you need. At the very least, it'll get you started and
enable you to see how useful this tool is without spending several lunch hours
getting it to do something! You can see a shot of xmeter as I use it at right.

It's difficult to find any information regarding the xmeter project's status or who maintains the source. In fact, it's difficult to even find a tarball of the source, so I've made one available on my site.

A new look at an old friend

I recently took an informal survey of the administrators I know in different
environments to see how they deal with the problem of sending commands to
multiple hosts simultaneously. The answers varied. Some relied on NFS -- config
file changes happened on an NFS server, and take effect immediately on
any hosts mounting that partition for their config information (I didn't ask
about restarting the services!). Others had written complex Perl scripts to get
the job done. Still others used commercial products.
What everyone else uses is only relevant to me in that it shows that no matter
what environment you work in, you need this ability!

Recently, I discovered
that there was a very workable solution already installed on my KDE desktop
machine -- Konsole!

Konsole -- click to enlarge

Typically, I try not to push tools that are tied to a particular desktop
environment. I use a different desktop environment on each machine I own, and
switch those around on a semi-monthly basis. I don't consider myself a "GNOME
user" or a "KDE user" -- I'm a Linux user, the desktop is secondary. That said,
Konsole has been running fine for me in a number of environments, the lone
exception being blackbox. I'll live.

The great feature of Konsole that I'm hooked on is the little toggle available
in newer versions that says "send input to all sessions." This lets you open up
any number of tabs in a Konsole window, ssh into all the hosts you want, toggle on this
option, and start typing commands that will be run on all of the machines for
which there is an open session! It will not interfere with sessions running in
other open Konsole windows, by the way.

While this can be useful for performing complex
tasks on a number of identical machines (I once editied four grub.conf files
simultaneously without error using vi), it's also a nice complement to other
monitoring tools. For example, xmeter may alert me to the fact that load is
getting out of hand on a particular server or set of servers, and I can run the
top command on all of them simultaneously to figure out which the troublemaker
is. A shot of Konsole running on my desktop appears above. Note the tab with
the little broadcast antenna icon on it -- that's the one you type in if you
wanted to distribute a command to all of the tabs in the window. If
you type in any other tab in the window, output is not sent to the other tabs.

Brian Jones is the founder of linuxlaboratory.org, and has worked as a *nix
systems, network, and database administrator for the past six years. He
currently works for the Computer Science department at Princeton University.

Click Here!