Log messages contain information about the events happening on the hosts. Monitoring system events is essential for security and system health monitoring reasons. A well-established log management solution offers several benefits to an organization. It ensures that computer security records are stored in sufficient detail, and provides a simple way to monitor and review these logs. Routine log reviews and continuous log analysis help to identify security incidents, policy violations, or other operational problems.
Logs also often form the basis of auditing and forensic analysis, product troubleshooting and support. There are also several laws, regulations and industrial standards that explicitly require the central collection, periodic review, and long-time archiving of log messages. Examples of such regulations are the Sarbanes-Oxley Act (SOX), the Basel II accord, the Health Insurance Portability and Accountability Act (HIPAA), or the Payment Card Industry Data Security Standard (PCI-DSS).
Built around the popular syslog-ng Premium Edition (syslog-ng PE) application used by thousands of organizations worldwide, the syslog-ng Store Box (SSB) brings you a powerful, easy-to-configure appliance to collect and store your logs. Using the features of the latest syslog-ng PE to their full power, SSB allows you to collect, process, and store log messages from a wide range of platforms and devices.
All data can be stored in encrypted and optionally time stamped files, preventing any modification or manipulation, satisfying the highest security standards and policy compliance requirements.
The syslog-ng Store Box (SSB) appliance is useful for everyone who has to collect, store, and review log messages. In particular, SSB is invaluable for:
-
Central log collection and archiving: SSB offers a simple, reliable, and convenient way of collecting log messages centrally. It is essentially a high-capacity log server with high availability support. Being able to collect logs from several different platforms makes it easy to integrate into any environment.
-
Secure log transfer and storage: Log messages often contain sensitive information and also form the basis of audit trails for several applications. Preventing eavesdropping during message transfer and unauthorized access once the messages reach the log server is essential for security and privacy reasons.
-
Policy compliance: Many organization must comply with regulations like the Sarbanes-Oxley Act (SOX), the Basel II accord, the Health Insurance Portability and Accountability Act (HIPAA), or the Payment Card Industry Data Security Standard (PCI-DSS). These regulations often have explicit or implicit requirements about log management, such as the central collection of log messages, the use of log analysis to prevent and detect security incidents, or guaranteeing the availability of log messages for an extended period of time — up to several years. SSB helps these organizations to comply with these regulations.
-
Automated log monitoring and log pre-processing: Monitoring log messages is an essential part of system-health monitoring and security incident detection and prevention. SSB offers a powerful platform that can classify tens of thousands of messages real-time to detect messages that deviate from regular messages, and promptly raise alerts. Although this classification does not offer as complete an inspection as a log analyzing application, SSB can process many more messages than a regular log analyzing engine, and also filter out unimportant messages to decrease the load on the log analyzing application.
This chapter discusses the technical concepts of syslog-ng Store Box (SSB).
The syslog-ng Store Box (SSB) appliance is a log server appliance that collects, stores and monitors log messages sent by network devices, applications and computers. SSB can receive traditional syslog messages, syslog messages that comply with the new Internet Engineering Task Force (IETF) standard (RFC 5424-5428), eventlog messages from Microsoft Windows hosts, as well as SNMP messages.
Figure 1: The philosophy of the syslog-ng Store Box
Clients can send messages to SSB using their own logging application if it supports the BSD-syslog (RFC 3164) or the IETF-syslog (RFC 5424-5428) protocol, or they can use the syslog-ng Premium Edition (syslog-ng PE) application to act as the log-forwarding agent of SSB.
The main purpose of SSB is to collect the logs from the clients and store them on its hard disk. The messages are stored in so-called logspaces. There are two types of logspaces: the first stores messages in traditional plain-text files, while the second one uses a binary format that can be compressed, encrypted, and time stamped.
You can also define multiple logspaces, remote logspaces, and configure filtered subsets of each logspace. A multiple logspace aggregates messages from multiple SSB appliances (located at different sites), allowing you to view and search the logs of several SSBs from a single web interface without having to log on to several different interfaces. Remote logspaces, on the other hand, enable you to access and search logspaces (including filtered logspaces) on other SSB appliances. Filtered logspaces allow the creation of a smaller, filtered subset of the logs contained in an existing local, remote or multiple logspace.
The syslog-ng PE application reads incoming messages and forwards them to the selected destinations. The syslog-ng PE application can receive messages from files, remote hosts, and other sources.
Log messages enter syslog-ng PE in one of the defined sources, and are sent to one or more destinations. In the case of the clients, one of the destinations is the syslog-ng Store Box. The destinations on the SSB can be logspaces or remote servers, such as database servers or log analyzing engines.
Sources and destinations are independent objects, log paths define what syslog-ng PE does with a message, connecting the sources to the destinations. A log path consists of one or more sources and one or more destinations: messages arriving to a source are sent to every destination listed in the log path. A log path defined in syslog-ng PE is called a log statement.
Optionally, log paths can include filters. Filters are rules that select only certain messages, for example, selecting only messages sent by a specific application. If a log path includes filters, syslog-ng PE sends only the messages satisfying the filter rules to the destinations set in the log path.
SSB is configured by an administrator or auditor using a web browser.
