By

In this article, I'll talk about using databases for authentication, rather than the standard text-file htpasswd files.

There are basically three reasons to use a database, rather than a text file, to store data.

The first reason is speed. Accessing data stored in a database is much faster than accessing data stored in a text file. A database is designed for rapid location of information. A text file, you have to read through each record until you find what you are looking for.

The second reason is ease of data retrieval. A database--at least, a decent database--provides you with a language (usually SQL) for querying the database for specific information.

The third reason is data integrity. Since a database handles a lot of things for you, which you would have to handle for yourself when using a text file, you are less likely to screw up your data, and lose information, when using a real database.

Why Use a Database for Authentication?

All of the above three reasons are important when selecting a method of authentication on your Apache server. If you're running a site with a very small number of user accounts, it may not be worth the hassle to try to use a database for your authentication. But, as your list of users grows, these things will become real assets.

As your list of users grows, it takes proportionately longer to find a given user in the password file. Past a certain number of people (about 2,000, in my experience) the look-up just takes too long. Users that are listed at the bottom of the file will just be denied access, because Apache gives up looking for them before it can get that far in the file.

Managing the user lists is easier also. Rather than trying to open a large text file, and scroll through it looking for a name, you can use database queries to find the user you're looking for, and change their password, or remove them, or add a new user.

If you let your users change their password, then you are at risk of corrupting your data. Consider the situation where two users try to exit their password at nearly the same time. User A loads the file into memory, and changes their password, and starts to write the file back to disk. At that moment, user B loads the file into memory to look for their password. Oops. The file has not been written back to disk yet, so user B only gets part of the file. When they write the file back to disk, most of the users mysteriously disappear. Actually, you'll (hopefully) implement some kind of file locking to avoid this completely, but using a database removes the concern completely.

OK, So How Do I Do This?

There are several Apache modules that let you use a database for your authentication. I'm going to talk about just two of them: mod_auth_db and mod_auth_mysql. I'll talk about mod_auth_db this week and leave mod_auth_mysql for next week, since there's a little more to say about it.

mod_auth_db

mod_auth_db lets you keep your usernames and passwords in DB files.

If you compiled Apache with mod_so enabled, enabling mod_auth_db should be just a matter of editing your httpd.conf file and uncommenting the line that refers to mod_auth_db. This should look something like:

        LoadModule db_auth_module     libexec/mod_auth_db.so

and, then ...

        AddModule mod_auth_db.c

Otherwise (if you don't have mod_so) you'll need to recompile Apache, and enable mod_auth_db. Since we talked about this in an earlier article, I'll not cover that here.

Wait a Second--What's a DB File?

Berkeley DB files are just one type of database files. They (usually) contain just key/value pairs, and so are rather limited in how much "real" database functionality you can get out of them (although there are some pretty slick extensions to them) but for HTTP authentication, a key/value pair is exactly what you want to store.

If you want to read more about DB files, you should look at the Sleepycat Software Web site, at http://www.sleepycat.com/ Sleepycat maintains the DB library and has some documentation about DB.

Protecting a Directory

Once you have compiled the mod_auth_db module, and loaded it into your web server, you'll find that there's very little difference between using regular authentication and using mod_auth_db authentication. In your .htaccess file, you'll need something like:

        AuthName        "Members Only"
        AuthType        Basic
        AuthDBUserFile  /usr/local/apache/passwd/passwords.dat
        AuthDBGroupFile /usr/local/apache/passwd/passwords.dat
        require user rbowen

Now, users accessing the directory will be required to authenticate against the list of valid users who are in /usr/local/apache/passwd/passwords.dat.

A Few Caveats

Well, there are a few different ways to get usernames/passwords in the DB file. And a few caveats are necessary here.

First, there are several different implementations of DB, with slightly different names. While I won't go into the gory details here (mostly because I don't know them all) suffice it to say that you may need to experiment some in order to get things working the way that you think they should. It's worth the effort, but be warned.

Secondly, just to confuse things a little further (at least in my mind) on Linux, two of the implementations (DB and DBM) which are usually different on other platforms, are the same.

Third, you'll find, as part of the standard Apache distribution, another module, called mod_auth_dbm, which works with DBM files, rather than DB files. Perhaps you see why I get confused sometimes.

And, finally, because there is so much platform dependency in these DB implementations, you'll find that a DB file (or DBM file) generated on one system may or may not work on another.

So, after all those caveats, you may be wondering if this is really worth it? Well, it is. These are things that might happen, but in practice (at least in my experience) seldom do.

How Do I Get Users Into the File?

Well, there's a tool that comes with Apache, called dbmmanage. You'll find it in the bin directory of wherever you installed Apache. (or, if you installed with a package manager, wherever it thought was a good place to put it. It might even be in your path.)

You'll find full documentation for dbmmanage by typing man dbmmanage, or various places online, like http://www.rt.com/man/dbmmanage.1.html It's simple to use, and you can use it to add or remove users from your password file one at a time.

If you are going to be doing more with these files, you will probably want something a little easier to automate. Perhaps the best tool for this will be Perl, using the DB_File module. The technique that is used with this module is a tied hash, which, simplified, means that the module causes the file to act like a hash, so that modifying the hash directly changes the DB file. Pretty cool.

The following Perl code, for example, will add a user rbowen, with password mypassword, to your password file:

        use DB_File;
        tie %database, 'DB_File', "passwords.dat"
                or die "Can't initialize database: $!\n";

        $username = 'rbowen';
        $password = 'mypassword';
        @chars=(0..9,'a'..'z');
        $salt = '', map { $chars[int rand @chars] } (0..1);

        $crypt = crypt($password, $salt);
        $database{$username} = $crypt;

        untie %database;

Passwords are stored in Unix crypt format, just as they were in the "regular" password files. The 'salt' that is created in the middle there is part of the process, cenerating a random starting point for that encryption. If enough people care, I'll explain this Perl code in a little more detail. Otherwise, just trust me, it works. I copied it from a web site that actually works. Of course, in the real world, the username and password are read from a web form, or something like that.

What About Groups?

In last week's article, we talked about putting users into groups and requiring a particular group of users. You can do the same thing with mod_auth_db, it just works a little differently. You'll notice that in my sample configuration, above, I had the following lines:

        AuthDBUserFile  /usr/local/apache/passwd/passwords.dat
        AuthDBGroupFile /usr/local/apache/passwd/passwords.dat

The user file and group file are pointing at the same location. What's up with that? It turns out that mod_auth_db stores both types of information in the same file.

Because DB files, as I mentioned early on in this article, just store a key/value pair, something has to be done to work around this limitation. What the authors of mod_auth_db decided to to was to put the group name in as part of the value, separated from the password by a colon.

So, if you were still using the Perl code above, you'd replace the line:

        $database{$username} = $crypt;

with

        $database{$username} = "$crypt:$group";

or something to that effect. You can specify more than one group by listing the groups, separated by commas.

I'm not aware of any nice way to do this with dbmmanage.

Once you have your passwords and groups in the file, you can require a group in the regular way:

        require group administrators

This is not the only way to do this, it's just the way that I do it. You can also have a separate group file, just like you do with regular text file authentication. If you ahve a separate group file, it would contain a list of username:group pairs. Again, you can have more than one group per username: just list them as a comma-separated list. And, as with the other method, I'm not aware of any nice way to do this with dbmmanage.

What about Microsoft Windows?

I actually don't know the answer to this one. I know that there are Windows implementations of DB files, so I don't know of any particular reason for it not to work. But I've never tried it myself. I'd love to hear from one of you telling me that you have it working on Windows, and telling me what you did to get it working.

Summary

mod_auth_db allows you to keep your users, passwords, and groups, in DB files rather than text files. You get all the usual benefits of using a database rather than a text file--faster access and better data reliability, and consequently, you can have larger user lists than you could support with a text file of users.

Future columns

Please let me know if there are other topics that you'd like for me to talk about in future columns. You can send your suggestions to me at And please let me know what you think of my columns, at that same address. Many thanks for reading down this far!

--Rich