Basics of Apache Logging: A Definitive Guide

Share on facebook
Share on twitter
Share on linkedin
Share on email

The need for an easy-to-use platform to log application data is exploding. Everyone wants to capture data about their users and their products. Apache Logging lets you do that easily, reliably, and at scale. This post captures everything related to including configuration and Apache log examples. Read on.

What is Apache Logging?

Apache Logging is an open-source project created to allow users to examine their logs efficiently. Apache Logging allows users to extract data from their log files and store it in other formats like CSV or XML. It also provides functions for comparing two different versions of the same log file.

The Apache Logging project is a software library that implements a logging service. It provides developers with a way to create and control logs, enabling them to see what is happening with the code.

Apache Logging services provide access to logs in a way that is easy for humans to read. The logs are stored in log files, which consist of messages showing the date and time, the server’s hostname, and the IP address of any client who requests a document.

Pick any Apache logs example you will find that they are used for debugging, tracing, monitoring and diagnostics. The logs can be used for testing, performance tuning, system administration, and even security analysis.

The Apache Logging project is the official successor of the Jakarta Commons Logging project. This open-source project contains log4j, a widely used tool for logging application behavior. It also contains the SLF4J API, which provides a framework for other loggers and allows them to be plugged into applications that use various other libraries, such as Apache Commons Logging or Google’s Guava.

How do I Enable Apache Logging

Apache Logging is typically a mechanism in a web server that allows users to track server activity. It can be enabled on your web server. Once enabled, it will start to log all the HTTP requests made by your visitors. It is a very handy feature of the Apache Webserver. It logs all the requests and responses and provides the ability to analyze the traffic in real-time. This helps you troubleshoot issues in your system quickly.

To enable Apache Logging, you need to configure a few directives in the webserver configuration file.

To enable Apache HTTP access logs, first you need to open HTTP configuration of Apache at /Applications/MAMP/conf/apache/httpd.conf

Next, find the code

#CustomLog logs/access_log combined

Replace this code with

CustomLog /Applications/MAMP/logs/apache_access_log combined

This will make sure all your access logs are logged in your default log directory, with the “Log Format” named “Combined”. This log format has some standard conventions.

Once done, restart your Apache server with the MAMP widget. You can also restart using the command line

$ /Applications/MAMP/bin/apache2/bin/apachectl restart

It is important to remember that if you enable the log directory, all your data will be written to /Applications/MAMP/Library/logs/access_log. This is not desirable. It is always ideal to store the access logs to /Applications/MAMP/logs/. This is where you could find MySQL, Apache error log, and PHP logs.

Types of Apache Logs 

Apache log structure is very flexible and easy to manage. There are two types of logs: Access logs and Error logs. 

Access Log

This is where all the information about the requests coming to the webserver is noted. Apache access log response time is also measured. The information can be anything like pages visited by the audience, requests success rate, and time is taken for the server to respond to the requests. To manage request logging, you need to be familiar with three configuration directories: LogFormat, CustomLog, and TransferLog.

Various other directories were available, but as Apache kept upgrading, these directories were deprecated. CustomLog can now achieve all the functions that these directories do. Few deprecated directories are RefererLog, CookieLog, RefererIgnore, and AgentLog.

Error Log

The information on errors encountered by the server during processing is found in the error log. It contains information on events unrelated to request serving and includes the diagnostics information about the server. The error log also contains the information that the access log doesn’t. Here is some log information that the error log offers:

  • Different informational messages
  • Critical Events
  • Errors that occurred during request servicing (status 400-503)
  • Standard error output
  • Startup and shutdown messages

The error log has a standard format. Every line contains three fields: time, error level, and messages. You can also get some raw data in the error logs during some rare instances. These logs are created by using the ErrorLog directory configuration. 

Log Locations

The storage location of the error log and access log files depends on your operating system. Both these files are stored as separate entities on the server. Let’s have a look at the default storage location for various operating systems. 

Note: To change the apache log directory, use the #grep command.

  • Linux Mint / Debian / Ubuntu

For the unencrypted sites, the httpd log file location in Linux is /etc/apache2/sites-available/000-default.conf. Similarly for the encrypted sites with SSL/TLS protection, the httpd log file location in Linux is /etc/apache2/sites-available/default-ssl.conf.

Here are the default derivatives for Linux OS –

SettingConfig FileValue/Path
Access Log/etc/apache2/sites-available/000-default.confCustomLog ${APACHE_LOG_DIR}/access.log combined
Log Level/etc/apache2/apache2.confWarnLogFormat “%v:%p %h %l %u %t “%r” %>s %O “%{Referer}i” “%{User-Agent}i”” vhost_combinedLogFormat “%h %l %u %t “%r”
Error Log/etc/apache2/apache2.confwarnErrorLog ${APACHE_LOG_DIR}/error.log
Custom Log/etc/apache2/conf-available/other-vhosts-access-log.confCustomLog ${APACHE_LOG_DIR}/other_vhosts_access.log vhost_combined
Log Format/etc/apache2/apache2.conf%>s %O “%{Referer}i” “%{User-Agent}i”” combinedLogFormat “%h %l %u %t “%r” %>s %O” commonLogFormat “%{Referer}i -> %U” refererLogFormat “%{User-agent}i” agent
  • CentOS / RedHat / Fedora

The main configuration file for the Red Hat distribution is located at /etc/httpd/conf/httpd.conf. The additional Virtual host config files can be placed in the directory /etc/httpd/conf.d. This directory is automatically read at the start by the server. Here are the default directives.

SettingConfig FileValue/Path
Access Log/etc/httpd/conf/httpd.conf/var/log/httpd/access_log
Error Log/etc/httpd/conf/httpd.conf/var/log/httpd/error_log
Log Level/etc/httpd/conf/httpd.confwarn
Custom Log/etc/httpd/conf/httpd.confCustomLog “logs/access_log” combined
  • OpenSUSE

Similarly, for the OpenSUSE Operating system, the default configuration for encrypted sites can be found at /etc/apache2/default-vhost-ssl.conf. The default virtual host config for the unencrypted sites can be found at /etc/apache2/default-vhost.conf. Here are the default directives – 

SettingConfig FileValue/Path
Access Log/etc/apache2/sysconfig.d/global.conf/var/log/apache2/access_log
Custom Log/etc/apache2/sysconfig.d/global.confCustomLog /var/log/apache2/access_log combined
Error Log/etc/apache2/httpd.conf/var/log/apache2/error_log
Log Level/etc/apache2/sysconfig.d/global.confwarn

Configuring Apache Error and Access Logs 

In the Apache framework, you have high flexibility to adjust the logging behaviour both globally or for each file of the vhost. There are various directives that you can use to change the Apache log directory behaviour. The most common are log level and log format directives.

Log Level directive

The main feature of the log level directive is to determine the minimum security level for the events that are logged to a specific destination. The importance level of an event can range from “Emerg” to “Trace8”. This typically represents the severity level. An event with the “Emerg” level might lead to instability, whereas the “Trace8” level provides trace level messages. Apache change log level can be altered according to your requirement.

Log Format

The layout and formatting of log events are controlled by the Log format directive. The default Apache log format is CLF (Common Log Format). But Apache gives you the flexibility to change fields included in each log by enabling you to specify your format string. These are a few default CLF:

Log Format “%h %l %u %t \”%r\” %>s %b”

The format string is represented in the first parameter. It indicates the information regarding the log file and the written format of the log file. To decipher the log format, refer to the documentation provided by Apache. These are a few standard logging strings formats – 

Format StringDescription
%%Percentage Sign
%…aRemote IP Address
%…ALocal IP Address
%…BBytes size for response (Excluding HTTP)
%…bBytes size for response (Excluding HTTP), Instead of Zero a dash (-) is used
%…DTime to serve (microseconds)
%…fName of the File
%…hRemote host
%…HRequest protocol
%…PProcess ID

These five fields are recommended as they are crucial for troubleshooting issues and monitoring server health – 

  1. %>s: Requests HTTP status code. The final request after internal redirection is shown.
  2. %U: The requested URL path excluding the additional query string is displayed.
  3. %a: This is used to identify the traffic from a particular source. It displays the IP of the client who is making the request.
  4. %T: Time taken to process the request in seconds. It is useful to measure the speed of the site.