Database support in Count 2.6

Database Support

From Count version 2.6+, the counter is able to store data in a database. It supports any of the Berkeley Db, GNU gdbm and ndbm databases. I suggest you use Berkeley Db for robustness and reliability . Note about Berkeley Db, the counter only supports version 1.85 or 2.x compiled with 1.85 API support. If you the get latest Berkeley db, please read docs/build/unix.html to know how to compile it with 1.85 API compatibility. The counter will not work Berkeley db 2.x API. If you have db 2.x, at counter link time, you will get an error message that the function dbopen() is not found. Try compiling the counter with default settings first. Your system will be probed for Berkeley db first, then GNU gdbm and lastly ndbm.

Data storing algorithm

The Counter data storing algorithm is shown below:

Count data storing algorithm

Supporting databases

The counter programs supports any of the Berkeley db, GNU gdbm, sdbm and ndbm databases. At compile time, the type of databases is decided and compiled with. Please look at the following table to compare the features of the various databases. It is clear that you should use Berkeley db.

Feature	NDBM	SDBM	GDBM	Berkeyley DB
Often comes with Unix	Yes ^a	No	No	No
Comes with Counter	No	Yes	No	No
Builds OK on Unix	N/A	Yes	Yes	Yes ^b
Code size	Varies^c	Small	Big	Big
Disk Usage	Varies^c	Small^c	Big	Ok^d
Speed	Varies	Slow^c	Ok	Fast^d
FTPable	No	Yes	Yes	Yes
Easy to build	N/A	Yes	Yes	Ok^e
Block size limits	4k	1k ^f	None	None ^e
Byte-order independent	No	No	No	Yes ^e
User-defined sort order	No	No	No	Yes ^e
Wildcard lookups	No	No	No	Yes ^e

Reference: Perl programming .., page?

The data is stored as key and value pair in the database. The key is the referer URL (the page where the counter is running) and the value is the hits:visitor_IP. Example:

Key=http://www.fccc.edu/users/muquit/Count.html
Value=23345:131.249.1.2

Utilities for manipulating databases

The counter program comes with several utilities to manipulate the counter database.

dumpdb - dump the Count database. The key and values are printed separated by "=>" Please look at utils/dumpdb directory. Usage: dumpdb db_file
dumpdb.pl - Perl script to do the same.
editdb - comprehensive program for the administrators to manipulate the counter database. It's located in utils/editdb directory.
count_admin.cgi - A Perl CGI program for users to manipulate their own counter values. Adminstrators can adminster the entire database. They adminstrators should configure the script allow users to manipulate their own entries only, but if needed certian users can be given more control.
rgbtxt2db - converts rgb.txt file a database for fast lookup. It is done automatically at install time. An example of rgb.txt file is:
```
255 250 250     snow
248 248 255     ghost white
248 248 255     GhostWhite
245 245 245     white smoke
245 245 245     WhiteSmoke
220 220 220     gainsboro
255 250 240     floral white
255 250 240     FloralWhite
253 245 230     old lace
253 245 230     OldLace
```
The program make the name of the color the key and the red, green, blue components the value. It also converts all the name to lower case. Please look at the Directory Structure for the location of rgbdb.

Authentication database
Authentication database is a database of URL WWW Homepage Access Counter uses. If strict_mode=yes in count.cfg file, then anyone wants to put a counter in a web page, the URL of the web page must be in this database. If the URL is not in the auth database, then the user will get a message like below:

Putting URL is authentication database is analogous to creating data files for the users in older versions of counter. It gives the administrators some control.

Authentication database can be manipulated by the program editdb or by the CGI program count_admin.cgi.

Here's an example of using editdb, assuming you copied editdb somewhere in your path:



    $ editdb /usr/local/etc/Counter/db/Authdb 
    (Current database:Authdb)
    Command> edit http
    <*> Locking database
    <*> Unocking database
    <->Lock released.

    3 items matched.
       1. http://localhost/=>Authenticate localhost
       2. http://localhost/ldap/=>http://localhost/ldap/
       3. http://localhost/~muquit/=>muquit'spage
    Enter the number to edit, * to edit all, Q to quit: 3

    Edit Key (http://localhost/~muquit/): 
    Edit Value (muquit'spage): Muquit's page
    <*> Locking database
    http://localhost/~muquit/=>Muquit'spage
    * saved successfully!

    <*> Unocking database
    <->Lock released.
    (Current database:Authdb)
    Command> q

If you have lots of URLs to add, write them in a text file and load the file with editdb. The format of this file is:


    URL=>comment

Here, URL is the key and the comment is the value. Let put the following two lines in a file called db.txt,


http://www.fccc.edu/users/muquit/=>Auth entry for my homepage
http://www.fccc.edu/users/muquit/Count.html=>Auth entry for Counter page

Now add the file to the auth database,


    $ editdb -f db.txt /usr/local/etc/Counter/db/Authdb
    Locking Database
    Add:http://www.fccc.edu/users/muquit/=>auth_entry_for_my_homepage
    Add:http://www.fccc.edu/users/muquit/Count.html=>auth_entry_for_Counter_page
    Lock released.

You can use dumpdb to dump the database.

Advantages of using databases

A single database for the whole site. No need to create a file for each page.
Much less disk I/O, hence faster.
Easy to administer. Users can edit their own hits.

Disadvantages of using databases

Solely depends on HTTP_REFERER environment variable. If the browsers stop supporting this variable, the counter will break.
Counter will show different values if the host has aliases and accessed via aliases. For example, for the host www.fccc.edu, the local users can access the host as www, so http://www/ will be another key in the database. However, for the practical purposes it is ok as most of the hosts in the Internet is accessed by only one name.