Installing AWStats to view offline logs.

AWStats is a web server log parsing tool usually used to parse live log files on a server. However if you have a downloaded log file and just wish to parse that instead it's easy to do with AWStats.

Installing AWStats

First make sure you have the package installed:

$ rpm -qa|grep awstats
awstats-6.5-5.fc5

If you don't you can install it from the AWStats website or by yum install awstats since it's part of the Fedora Extras repository.

On Fedora it will install the main files in these locations.

Download the log files.

Next make a directory where you wish to keep you downloaded log files, in my case I'll use a folder called apache_logs under my home folder.

$ mkdir ~/apache_logs

Download the log file you wish to parse and save it (or move it) into the folder made previously. With my hosts log files they are archived every day with a new archive for every month, to enable AWStats to parse these files all together we need to make one large log file from the individual files.

In my case I untar all the files into my apache_logs folder and rename them by source and month number so they look something like this:

$ ls -1
apachelog_3
apachelog_4
apachelog_5
apachelog_6
apachelog_7

Of course the name you pick is completely up to you but doing it by ascending number helps when joining the files into one. Next we need to put these files into one main file using the cat command:

$ cat apachelog_* > apachelog_main

This command joins all the files like 'apachelog_' and send the output to a new file called apachelog_main, this is the file we shall now analyse. By using ascending numbers for the files we ensure they are appended starting at the oldest file with newer records below, just like the real web log format.

Edit the config file

Go to the location of the AWStats configuration files, on Fedora Core they are located in /etc/awstats.

$ cd /etc/awstats

Copy the file awstats.model.conf and rename it to a chosen AWStats profile name, this is of the format awstats.your-name.conf. I'll name it awstats.bobpeers.conf

Into this file edit or check the following information is correct:

LogFile="/home/user/apache_logs/apachelog_main"
LogType=W
LogFormat=1
LogSeparator=" "
SiteDomain="bobpeers.com"
DNSLookup=2
DirData="/var/lib/awstats"
DirCgi="/awstats"
DirIcons="/awstatsicons"
AllowFullYearView=3

Most importantly the LogFile should point to the main file we created at the start in our apache_logs folder (apachelog_main).

Update AWStats

Next we need to run the update process, open a terminal and change directory to our AWStats installation cgi-bin:

$ cd /usr/share/awstats/wwwroot/cgi-bin

To run the update process use the command below as root (since we need write access to /var/lib/awstats)

$ sudo ./awstats.pl -config=bobpeers update
Update for config "/etc/awstats/awstats.bobpeers.conf"
With data in log file "/home/bobuser/apache_logs/apachelog_main"...
Phase 1 : First bypass old records, searching new record...
Searching new records from beginning of log file...
Phase 2 : Now process new records (Flush history on disk after 20000 hosts)...
Jumped lines in file: 0
Parsed lines in file: 219016
 Found 57 dropped records,
 Found 0 corrupted records,
 Found 0 old records,
 Found 218959 new qualified records.

Replace bobpeers with the AWStats profile name you chose when naming the config file above. After completing AWStats creates it's own database located in /usr/lib/awstats.

Make a Virtual Host for AWStats

If you don't have any virtual hosts set up make a file called vhosts.conf in your /etc/httpd/conf.d/ folder and add the following lines to it. If you use the main httpd.conf file for your virtual hosts then add this to that file instead.

<VirtualHost 127.0.0.1>
   DocumentRoot /usr/share/awstats/wwwroot
   ServerName logfiles
   Alias /awstatsclasses "/usr/share/awstats/wwwroot/classes/"
   Alias /awstatscss "/usr/share/awstats/wwwroot/css/"
   Alias /awstatsicons "/usr/share/awstats/wwwroot/icon/"
   ScriptAlias /awstats/ "/usr/share/awstats/wwwroot/cgi-bin/"
</VirtualHost>

Next we need to add an entry into our Hosts file so your browser knows that the servername 'logfiles' is actually 127.0.0.1 or localhost. Open the file /etc/hosts as root and add this line:

127.0.0.1		logfiles

Finally we need to allow Apache to access the wwwroot folder in the AWStats install folder.

$ chcon -R user_u:object_r:httpd_sys_content_t /usr/share/awstats/wwwroot
$ chmod -R 755 /usr/share/awstats/wwwroot

The first line allows tells SELinux to let Apache access this folder, if you turn off SELinux there's no need for this command.

Alternatively you could always copy the files to a different location and set up the Virtual Host to look there instead.

If Apache is already running you will need to restart it to pick up the new virtual host:

$ /sbin/service httpd restart

Finally point your browser to the location http://logfiles/awstats/awstats.pl?config=bobpeers replacing bobpeers with your chosen AWStats profile name to see the web logs parsed by AWStats.