March 10, 2006

My sysadmin toolbox

Author: Brent Durksen

I maintain a Web server using Apache 2, PHP, Perl, MySQL, and OpenSSL; an IMAP server running the up-and-coming RoundCube Webmail client; and a server for streaming MP3s. GNU Emacs, OpenSSH, TightVNC, and netstat are just a few of the tools I use to maintain my servers.

GNU Emacs

GNU Emacs is my editor of choice. Yes, I know a lot of people swear by Vim, which is more universally available, but Vim's shortcuts are just too unnatural to use with the Dvorak keyboard layout, which I prefer.

Being able to jump through a text file by character, word, sentence, paragraph, and page with just a few keystrokes saves a lot of time when editing long configuration files. I use Vim when I have no choice, but I feel much more at home in Emacs.

OpenSSH

I also use the OpenSSH suite extensively, almost always used in conjunction with GNU Screen. With OpenSSH I can do almost everything remotely that I could do at the physical terminal itself. I make sure I never send passwords over the Internet unencrypted, so I sleep better at night knowing SSH is keeping my data secure.

TightVNC

When the command line isn't enough, such as when I want to use Ethereal's GUI to analyze my network traffic, VNC does the trick. Using the TightVNC server through an SSH tunnel allows me to use those graphical tools without too much lag.

In the past I used the NoMachine NX server, which is significantly faster, but I grew frustrated at having to install the client on every computer from which I wanted to access my own machine. The TightVNC client for Windows is a standalone binary that does not require Administrator privileges for installation, meaning that I can access my server quickly from any Internet cafe, friend's house, and other locations.

Netstat

Netstat is another utility that I rely on heavily. Few days go by when I don't use the command sudo netstat -tap to let me know who's connected to my computer, and which processes are listening for network connections. Netstat is my first line of defense against intruders.

Alias

I get frustrated quickly when I'm working on another computer without my bash aliases. I've defined dozens of aliases; here's a small sample of the aliases I use regularly:

l = ls --auto-color
ll = ls -lah
la = ls -A
lt = ls -lhtr # give a long list of files, with the newest files last
em = emacs -nw
suem = sudo emacs -nw # quick way to edit config files as root
agi = sudo apt-get install
acs = apt-cache search
pid = ps wax|grep # search for a PID.

Defining aliases for commands or command combinations that you use frequently can save a lot of time and typing. See Keith Winston's article on aliases for an overview of setting up your own aliases.

Standard *nix utilities

I have written numerous little bash scripts using grep, cut, head, tail, and other standard Unix utilities, often chained together through a staggering number of pipes. The cut tool, in particular, is an interesting one to read up on if you've never used it before.

The cut utility allows you to separate a line into multiple fields separated by arbitrarily defined delimiters, such as commas, colons, or other characters commonly used to separate fields in text files. With cut, it's easy to parse a comma-separated value (CSV) file in a bash script. But cut has many other useful applications as well.

My server sits behind a NAT router, so finding out my public IP address is a non-trivial task. I can use curl to poll checkip.dyndns.org for my current address:

curl -s checkip.dyndns.org

The current IP check returns the information in this format: <html><head><title>Current IP Check</title></head><body>Current IP Address: 216.239.39.99</body></html>

Using cut, I can extract just the information that I need:

curl -s checkip.dyndns.org|cut -d ":" -f2|cut -d "<" -f1

That produces something a bit more readable: 216.239.39.99

See the cut man page for more information on its usage.

Secure locate

Secure locate, or slocate, is another utility I use quite often. While find is a great tool, if I don't know which directory a file is in, it can take forever to search an entire filesystem. Having an indexed search tool speeds things up dramatically. A search using find can take quite a while, but searching with slocate takes only a few seconds.

Display tasks with top

Another tool I use quite often is top. When using top, I have my screen split between two lists, one showing processes sorted by highest memory consumption, and the other showing those that are hogging the CPU. This lets me quickly identify which processes are slowing things down. I usually leave top running in a screen virtual terminal.

Staying secure with mod_ssl

I also use mod_ssl so I can encrypt traffic between my server and myself. I run a secure server that indexes certain sensitive directories (/var/log, for example), gives them reasonably obscure aliases, and password-protects them so I can view them on the Web. I can then quickly check the status of my server from anywhere in the world using only a Web browser.

I can even use CGI to access the output of programs like netstat or dmesg in my browser. A word of caution, however: if you plan to allow access to these directories from any public computer, think about using an authentication method that does not allow the browser to store your password -- most browsers do temporarily remember your password for sites that use Apache's basic authentication method. You want to ensure that your login information is not remembered by any browser when you walk away.

Looking for help

For situations when I need to learn more about a command or utility, I find that man answers about 90% of my questions, and Google answers the rest.

Let us know about your most valuable utilities and how you use them. There need not be 10 of them, nor do they need to be in order, and if we publish your work, we'll pay you $100.

Click Here!