Chat now with support
Chat with Support

syslog-ng Store Box 5.0.3 - Administration Guide

Preface Introduction The concepts of SSB The Welcome Wizard and the first login Basic settings User management and access control Managing SSB Configuring message sources Storing messages on SSB Forwarding messages from SSB Log paths: routing and processing messages Configuring syslog-ng options Searching log messages Searching the internal messages of SSB Classifying messages with pattern databases The SSB RPC API Troubleshooting SSB Security checklist for configuring SSB About us Third-party contributions

Who uses SSB

SSB is useful for everyone who has to collect, store, and review log messages. In particular, SSB is invaluable for:

  • Central log collection and archiving: SSB offers a simple, reliable, and convenient way of collecting log messages centrally. It is essentially a high-capacity log server with high availability support. Being able to collect logs from several different platforms makes it easy to integrate into any environment.

  • Secure log transfer and storage: Log messages often contain sensitive information and also form the basis of audit trails for several applications. Preventing eavesdropping during message transfer and unauthorized access once the messages reach the log server is essential for security and privacy reasons.

  • Policy compliance: Many organization must comply with regulations like the Sarbanes-Oxley Act (SOX), the Basel II accord, the Health Insurance Portability and Accountability Act (HIPAA), or the Payment Card Industry Data Security Standard (PCI-DSS). These regulations often have explicit or implicit requirements about log management, such as the central collection of log messages, the use of log analysis to prevent and detect security incidents, or guaranteeing the availability of log messages for an extended period of time — up to several years. SSB helps these organizations to comply with these regulations.

  • Automated log monitoring and log pre-processing: Monitoring log messages is an essential part of system-health monitoring and security incident detection and prevention. SSB offers a powerful platform that can classify tens of thousands of messages real-time to detect messages that deviate from regular messages, and promptly raise alerts. Although this classification does not offer as complete an inspection as a log analyzing application, SSB can process many more messages than a regular log analyzing engine, and also filter out unimportant messages to decrease the load on the log analyzing application.

The concepts of SSB

This chapter discusses the technical concepts of SSB.

The philosophy of SSB

The syslog-ng Store Box (SSB) is a log server appliance that collects, stores and monitors log messages sent by network devices, applications and computers. SSB can receive traditional syslog messages, syslog messages that comply with the new Internet Engineering Task Force (IETF) standard (RFC 5424-5428), eventlog messages from Microsoft Windows hosts, as well as SNMP messages.

Figure 1: The philosophy of the syslog-ng Store Box

Clients can send messages to SSB using their own logging application if it supports the BSD-syslog (RFC 3164) or the IETF-syslog (RFC 5424-5428) protocol, or they can use the syslog-ng Premium Edition application to act as the log-forwarding agent of SSB.

The main purpose of SSB is to collect the logs from the clients and store them on its hard disk. The messages are stored in so-called logspaces. There are two types of logspaces: the first stores messages in traditional plain-text files, while the second one uses a binary format that can be compressed, encrypted, and timestamped.

You can also define multiple logspaces, remote logspaces, and configure filtered subsets of each logspace. A multiple logspace aggregates messages from multiple SSBs (located at different sites), allowing you to view and search the logs of several SSBs from a single web interface without having to log on to several different interfaces. Remote logspaces, on the other hand, enable you to access and search logspaces (including filtered logspaces) on other SSB appliances. Filtered logspaces allow the creation of a smaller, filtered subset of the logs contained in an existing local, remote or multiple logspace.

The syslog-ng application reads incoming messages and forwards them to the selected destinations. The syslog-ng application can receive messages from files, remote hosts, and other sources.

Log messages enter syslog-ng in one of the defined sources, and are sent to one or more destinations. In the case of the clients, one of the destinations is the syslog-ng Store Box. The destinations on the SSB can be logspaces or remote servers, such as database servers or log analyzing engines.

Sources and destinations are independent objects, log paths define what syslog-ng does with a message, connecting the sources to the destinations. A log path consists of one or more sources and one or more destinations: messages arriving to a source are sent to every destination listed in the log path. A log path defined in syslog-ng is called a log statement.

Optionally, log paths can include filters. Filters are rules that select only certain messages, for example, selecting only messages sent by a specific application. If a log path includes filters, syslog-ng sends only the messages satisfying the filter rules to the destinations set in the log path.

SSB is configured by an administrator or auditor using a web browser.

Collecting logs with SSB


The following procedure illustrates the route of a log message from its source on the syslog-ng client to the syslog-ng Store Box.

Figure 2: The route of a log message

  1. A device or application sends a log message to a source on the syslog-ng client. For example, an Apache web server running on Linux enters a message into the /var/log/apache file.

  2. The syslog-ng client running on the web server reads the message from its /var/log/apache source.

  3. The syslog-ng client processes the first log statement that includes the /var/log/apache source.

  4. The syslog-ng client performs optional operations on the message, for example, it rewrites parts of the message or compares the message to the filters of the log statement (if any). If the message complies with all filter rules, syslog-ng sends the message to the destinations set in the log statement, for example, to the remote syslog-ng server.

    After that, the syslog-ng client processes the next log statement that includes the /var/log/apache source, repeating Steps 3-4.

  5. The message sent by the syslog-ng client arrives to a source set on the syslog-ng Store Box.

  6. The syslog-ng Store Box reads the message from its source and processes the first log path that includes that source.

  7. The syslog-ng Store Box processes the message and performs the following operations. Note that most of these operations are optional, but the order of the processing steps is fixed.

    1. Parse the message as a syslog message (unless message parsing is explicitly disabled for the source).

    2. Classify the message using a pattern database.

    3. Modify the message using rewrite rules (before filtering).

    4. Filter the messages, for example, based on sender hostname or message content. If the message does not match the configured filter, SSB will not send it to the destination.

    5. Parse the text of the message (that is, the ${MESSAGE} part) using a key-value parser or the sudo parser.

    6. Modify the message using rewrite rules (after filtering and other parsing).

    7. SSB sends the message to the destinations set in the log path. The destinations are local, optionally encrypted files on SSB, or remote servers, such as a database server.

  8. SSB processes the next log statement, repeating Steps 6-8.


    The syslog-ng application can stop reading messages from its sources if the destinations cannot process the sent messages. This feature is called flow-control and is detailed in Managing incoming and outgoing messages with flow-control.

Related Documents