Linux.com

Feature

The Eighth Commandment of system administration

By Brian Warshawsky on June 13, 2005 (8:00:00 AM)

Share    Print    Comments   

A system log is one of the most effective ways to monitor a server's health and underlying problems. Often before a major hardware or application crash takes place there are indicators of impending disaster within the syslog. As a good and attentive administrator, you should be reviewing your logs on a regular basis, but oftentimes these logs are forgotten due to other duties or important data is lost within pages of white noise telling about normal events.

VIII. Thou shalt not lose system logs when a server dies

When that impending disaster finally hits and the machine falls dead, it can be handy to have a copy of the log files stored safely somewhere else. Then you can immediately review the logs to see what you're dealing with without having to spend time trying to pull the logs off the server, or worse yet, rebuilding the server without knowing what caused it to crash. The functionality to do this is already built into the Linux kernel, and it is easy to implement.

The syslog daemon has several options you can specify. Among the most useful is the remote logging feature. To enable it on what will be your remote syslog server, go to<nobr> <wbr></nobr>/etc/rc.d/init.d/syslog and open it with your favorite text editor. Find the line that reads

SYSLOGD_OPTIONS="-m 0"

and modify it so that it reads

SYSLOGD_OPTIONS="-m 0 -r"

Then you must find and modify the same line in<nobr> <wbr></nobr>/etc/sysconfig/syslog in the same manner. Once you've done this, the last thing to do is add the syslog service into<nobr> <wbr></nobr>/etc/services as 514/udp. When you add it, make sure you add it below isakmp to ensure numerical order within the file. When all this is done, restart syslog by entering the command

#<nobr> <wbr></nobr>/sbin/service syslog restart

Once syslog restarts, your machine will be listening on port 514 for syslog messages from other hosts.

Now all you need to do is configure those hosts to broadcast their syslogs to the syslog server. To do so, open the<nobr> <wbr></nobr>/etc/syslog.conf in your favorite text editor and add the following line:

*.* xxx.xxx.xxx.xxx

where xxx.xxx.xxx.xxx is the IP address of the remote syslog server you just configured. Once you've done this, restart syslog and check the remote syslog server. There should be an entry at the bottom of its log telling you that syslog has started on the remote machine.

You can configure as many hosts as you want to communicate with the syslog server. The best part is that many devices, such as firewalls, wireless access points, printers, switches, and spam blockers, all allow for remote syslog. This technique lets you keep their log files in one central location.

The commandments so far:
I. Thou shalt make regular and complete backups
II. Thou shalt establish absolute trust in thy servers
III. Thou shalt be the first to know when something goes down
IV. Thou shalt keep server logs on everything
V. Thou shalt document complete and effective policies and procedures
VI. Thou shalt know what cable goes where
VII. Thou shalt use encryption for insecure services
VIII. Thou shalt not lose system logs when a server dies

Share    Print    Comments   

Comments

on The Eighth Commandment of system administration

Note: Comments are owned by the poster. We are not responsible for their content.

Plain text = Bad idea

Posted by: Anonymous Coward on June 14, 2005 04:42 AM
Sending the syslog data unencrypted over the wire is an extremely bad idea. Instead one should set up a stunnel (as in <a href="http://www.stunnel.org/examples/syslog-ng.html" title="stunnel.org">http://www.stunnel.org/examples/syslog-ng.html</a stunnel.org> for example).

#

Re:Plain text = Bad idea

Posted by: Anonymous Coward on June 14, 2005 10:13 PM
This is particularly ironic as the Seventh Commandment of system administration is 'Thou shalt use encryption for insecure services'. Can anyone say "D'oh!"?

#

Re:Plain text = Bad idea

Posted by: Anonymous Coward on June 15, 2005 01:25 AM
True enough, but doesn't stunnel only work on TCP connections, while most sysloggers only do UDP?

Fortunately, there are some syslog programs that support TCP, such as syslog-ng

#

Paths are for RH

Posted by: Anonymous Coward on June 28, 2005 01:12 AM
It should be noted that the paths described in this article (/etc/rc.d/init.d/syslog,<nobr> <wbr></nobr>/etc/sysconfig/syslog) are Redhat-specific.

#

This story has been archived. Comments can no longer be posted.



 
Tableless layout Validate XHTML 1.0 Strict Validate CSS Powered by Xaraya