These classnotes are depreciated. As of 2005, I no longer teach the classes. Notes will remain online for legacy purposes

UNIX03/Scanning And Monitoring System Logs

Classnotes | UNIX03 | RecentChanges | Preferences

One important task in the security world is to regularly check the log files. Often the daily activities of an administrator don't allow him the time to do this task and this can bring about problems.

The biggest problem is that these log files can become quite massive and there can be many of them. Just do an ls -l in /var/log and you will see that there is an awful lot of information your UNIX system logs for you.

Sometimes, a particular log file may explode when a very serious problem occurs, making the log files even larger, harder to parse, and possibly even consuming all of your system's resources. As an example, GNU Mailman (which we looked at in UNIX02, see UNIX02/Introduction To GNUMailman) has a subprocess that deals with delivery and archiving called "qrunner". qrunner has a problem when used with the buggy Pipermail archiver that can cause lost files, and result in the following entries in the qrunner log file to be added every minute:

 Jun 23 11:09:00 2003 (1045) Exception reading qfile:
   /home/mailman/qfiles/3aa6abd913e307865e693f91ca77198920bbe346
 [Errno 2] No such file or directory:
   '/home/mailman/qfiles/3aa6abd913e307865e693f91ca77198920bbe346.db'
 Jun 23 11:10:00 2003 (1051) Exception reading qfile:
   /home/mailman/qfiles/3aa6abd913e307865e693f91ca77198920bbe346
 [Errno 2] No such file or directory:
   '/home/mailman/qfiles/3aa6abd913e307865e693f91ca77198920bbe346.db'
 Jun 23 11:11:01 2003 (1053) Exception reading qfile:
   /home/mailman/qfiles/3aa6abd913e307865e693f91ca77198920bbe346
 [Errno 2] No such file or directory:
   '/home/mailman/qfiles/3aa6abd913e307865e693f91ca77198920bbe346.db'

Not only does this rapidly consume hard drive space, it also may bring the Mailman system to a grinding halt.

Additionally, since your system will log so much activity, and the logs can be so cryptic, it can be hard to look at a given log file (with hundreds if not thousands of lines) and disect it to find those fingerprints that point to a system problem or a cracker attack.

Thus, keeping up on all of the log files on your system is a difficult, and probably impossible task.... At least on your own.



Classnotes | UNIX03 | RecentChanges | Preferences
This page is read-only | View other revisions
Last edited June 28, 2003 2:53 am (diff)
Search:
(C) Copyright 2003 Samuel Hart
Creative Commons License
This work is licensed under a Creative Commons License.