Your Daily Source for Apache News and Information

By

Long ago, log files came in one format. It was called the common format, and you were pretty much stuck with it. Then came custom log file format, and it turned out to be such a good idea that even the common format was reimplemented as a custom log file format.

In this article, you'll find out how to make your log files look like whatever you want, and have whatever information in them that you want.

Overview

Here's the lightning overview for those of you that just want to get something working, and don't care about all the details. You'll need to look at the LogFormat and CustomLog directives. There are several examples in your default httpd.conf file.

LogFormat sets up a format and gives it a nickname by which you can refer to it. CustomLog sets up an actual log file, and indicates the format (by nickname, usually) that file will use.

LogFormat

The LogFormat directive sets up a log format, and a nickname by which you can refer to that format.

For example, in your default httpd.conf file, you'll find the following line:

     LogFormat "%h %l %u %t \"%r\" %>s %b" common

This directive creates a log format called "common", which is in the format specified in quotes. Each one of those letters means a particular piece of information, which is put into the log file in the order indicated.

The available variables, and their meanings, are listed in the documentation, and are reproduced below:

     %...a:          Remote IP-address
     %...A:          Local IP-address
     %...B:          Bytes sent, excluding HTTP headers.
     %...b:          Bytes sent, excluding HTTP headers. In CLF format
                     i.e. a '-' rather than a 0 when no bytes are sent.
     %...{FOOBAR}e:  The contents of the environment variable FOOBAR
     %...f:          Filename
     %...h:          Remote host
     %...H           The request protocol
     %...{Foobar}i:  The contents of Foobar: header line(s) in the request
                     sent to the server.
     %...l:          Remote logname (from identd, if supplied)
     %...m           The request method
     %...{Foobar}n:  The contents of note "Foobar" from another module.
     %...{Foobar}o:  The contents of Foobar: header line(s) in the reply.
     %...p:          The canonical Port of the server serving the request
     %...P:          The process ID of the child that serviced the request.
     %...q           The query string (prepended with a ? if a query string exists,
                     otherwise an empty string)
     %...r:          First line of request
     %...s:          Status.  For requests that got internally redirected, this is
                     the status of the *original* request --- %...>s for the last.
     %...t:          Time, in common log format time format (standard english format)
     %...{format}t:  The time, in the form given by format, which should
                     be in strftime(3) format. (potentially localised)
     %...T:          The time taken to serve the request, in seconds.
     %...u:          Remote user (from auth; may be bogus if return status (%s) is 401)
     %...U:          The URL path requested.
     %...v:          The canonical ServerName of the server serving the request.
     %...V:          The server name according to the UseCanonicalName setting.

In each case, the "..." indicates an (optional) condition. If the condition is met, then the particular. If the condition is ommitted, then the variable will be replaced with a "-" if it is not defined. I'll give some examples of this in a minute.

So, the LogFormat line above, from the default httpd.conf file, creates a log format called common, which contains the remote host, remote logname, remote user, the time of the transaction, the first line of the request, the status of the request, and the number of bytes sent. Which is pretty much what I went through in my last article.

Now, sometimes you'll only want a particlar piece of information logged if it is defined. These is what the "...", referred to above, provides for. If, between the % and the variable, you put one or more HTTP status codes, the variable will only be logged in the event that the request returns one of those status codes. So, if you're trying to keep a log of all the broken links on your site, you might have the following:

     LogFormat %404{Referer}i BrokenLinks

Conversely, if you want to log requests that don't match a particular code, put a ! in there:

     LogFormat %!200U SomethingWrong

CustomLog

Once you have set up one or more LogFormats, you just have to apply them to a particular log file. This is done with the CustomLog directive. You can set up as many log files as you like (well, not really, but you can set up a lot of them). Each one needs to specify a lof file location, and which LogFormat you want to use:

     CustomLog /var/log/httpd/bogus_log SomethingWrong
     CustomLog /usr/local/apache/logs/broken BrokenLinks
     CustomLog /usr/local/apache/logs/access_log common

Conclusion

That's really all there is to it. You can put just about any information in a log file, and format it just about any way you like.

The only disadvantage to doing this is that if you get some off the shelf log analysis application, it will assume that you are using common or combined log format, since those are the ones that are most widely in use.

Next Time

That's all 'til next time. In the next article, I'll talk about parsing your log files -- running some sort of analysis tool on them to get useful information out. That is the thing that your boss really wants. How many people looked at the web site yesterday? Where did they find out about us? Are they coming back, or just one-time visitors? And so on.

Send me a note at if you have any questions, or suggestions for another article. Or make a comment in the Talkback area.

Want to discuss logfiles with other Apache Today readers? Then check out the discussions at Apache Today Discussions.

All times are recorded in UTC. Linux is a trademark of Linus Torvalds. Powered by Linux 2.4, Apache 1.3, and PHP 4 Copyright INT Media Group, Incorporated All Rights Reserved. Legal Notices,  Licensing, Reprints, & Permissions,  Privacy Policy.