How to Do Application Logging Right is the best guidance I have come across to date. Co-written byAnton Chuvakin and Gunnar Peterson for the IEEE Security & Privacy Journal, the paper describes the problems with typical logging systems, what events need logging, and for those, what to include and exclude. They have also provided some broader guidance on log management and protection.

Previously, the most notable application security logging guidance existed buried rather deeply in the documentation for OWASP's ESAPI Java edition, the OWASP Logging Project, and more general guidance in NIST's SP 800-92 Guide to Computer Security Log Management.

If you read those in conjunction with the new paper, and perhaps Chuvakin's and Peterson's own comments, you'll be well up to speed.

The content of the "module", "object" and "action" fields will be dependent upon the degree of granularity required and how much additional event information is collected as additional details (e.g. stack trace, request headers, response body). I believe a transaction ID should always be included so that all events for a single request/response can be more easily correlated—this has a request scope rather than the session scope of a username/id. If I might suggest some other additional items for "what to include", I would also consider:

• host address (e.g. host name and domain, or server IPv4 or IPv6 address) which is useful if clustering is being used, or to confirm logs are from live rather than staging systems
• service (e.g. name, port and protocol)
• full actual entry point URL (protocol, full domain, port, path and further parameters)
• canonicalised entry point URL
• HTTP method (for web applications)
• responses seen by the user and/or taken by the application (e.g. status code, custom text messages, session termination, administrator alerts)
• analytical confidence in the event detection (low, medium, high or a numeric value).

Full request headers and possibly the response body may be worth collecting for some events. But ensure these are sanitised for sensitive input such as passwords, session cookies or credit card numbers.

I would also tend to use a severity scale (0=emergency, 1=alert, ..., 7=debug) rather than the suggested "priority" field, for consistency with syslog protocol. But the paper's authors note that whatever scale is used, it will be different for each organisation due to their own priorities and views on risk.

You may also want to consider how the integrity of the logged information can be determined.

Whatever you log, bear in mind you probably want it to be relatively human-readable, but also done in a way you can share the information with other systems. For the moment, consider Common Event Format(CEF). But Common Event Expression (CEE) is an ongoing collaborative effort to develop an event interoperability format summarised in a presentation, and in more detail in a white paper. The CEE web site includes a description of alternative approaches for sharing data from event producers.

See also my previous web application logging related posts How Much Logging, Monitoring and Alerting?, Security Logging Requirements, Testing the Audit Trail, Don't Stop the Attack (Too Soon), and Application Log Management and Analysis.

Happy application logging!