Collecting logs with SSB
The following procedure illustrates the route of a log message from its source on the syslog-ng Premium Edition (syslog-ng PE) client to the syslog-ng Store Box appliance.
Figure 2: The route of a log message
-
A device or application sends a log message to a source on the syslog-ng PE client. For example, an Apache web server running on Linux enters a message into the /var/log/apache file.
-
The syslog-ng PE client running on the web server reads the message from its /var/log/apache source.
-
The syslog-ng PE client processes the first log statement that includes the /var/log/apache source.
-
The syslog-ng PE client performs optional operations on the message, for example, it rewrites parts of the message or compares the message to the filters of the log statement (if any). If the message complies with all filter rules, syslog-ng PE sends the message to the destinations set in the log statement, for example, to the remote syslog-ng PE server.
After that, the syslog-ng PE client processes the next log statement that includes the /var/log/apache source, repeating Steps 3-4.
-
The message sent by the syslog-ng PE client arrives to a source set on the syslog-ng Store Box appliance.
-
The syslog-ng Store Box appliance reads the message from its source and processes the first log path that includes that source.
-
The syslog-ng Store Box appliance processes the message and performs the following operations. Note that most of these operations are optional, but the order of the processing steps is fixed.
-
Parse the message as a syslog message (unless message parsing is explicitly disabled for the source).
-
Classify the message using a pattern database.
-
Modify the message using rewrite rules (before filtering).
-
Filter the messages, for example, based on sender hostname or message content. If the message does not match the configured filter, syslog-ng Store Box(SSB) will not send it to the destination.
-
Parse the text of the message (that is, the ${MESSAGE} part) using a key-value parser or the sudo parser.
-
Modify the message using rewrite rules (after filtering and other parsing).
-
SSB sends the message to the destinations set in the log path. The destinations are local, optionally encrypted files on SSB, or remote servers, such as a database server.
-
SSB processes the next log statement, repeating Steps 6-8.
Managing incoming and outgoing messages with flow-control
This section describes the internal message-processing model of syslog-ng Premium Edition (syslog-ng PE), as well as the flow-control feature that can prevent message loss. To use flow-control, the flow-control option must be enabled for the particular log path.
The internal message-processing model of syslog-ng PE
-
The syslog-ng PE application checks the source for messages.
-
When a log message is found, syslog-ng PE reads the message.
-
The message is processed and put into the output buffer of the destination.
-
When the destination can accept the message, syslog-ng PE sends the message to the destination from the output buffer.
Flow-control
If the destination cannot send out messages, or not as fast as they arrive in the destination, the output buffer fills up. When the output buffer is full, the sources stop reading messages. This can prevent message loss.
If a message is successfully sent out from the destination, the source that sent that message starts reading logs again, until the destination buffer fills up.
Flow-control and multiple destinations
Using flow-control on a source has an important side-effect if the messages of the source are sent to multiple destinations. If flow-control is in use and one of the destinations cannot accept the messages, the other destinations do not receive any messages either, because syslog-ng PE stops reading the source. For example, if messages from a source are sent to a remote server and also stored locally in a file, and the network connection to the server becomes unavailable, neither the remote server nor the local file will receive any messages. This side-effect of the flow-control can be avoided by using the disk-based buffering feature of syslog-ng PE.
NOTE: Creating separate log paths for the destinations that use the same flow-controlled source does not help avoiding the problem.
Receiving logs from a secure channel
The syslog-ng Store Box (SSB) appliance receives log messages securely over the network using the Transport Layer Security (TLS) protocol (TLS is an encryption protocol over the TCP/IP network protocol).
TLS uses certificates to authenticate and encrypt communication, as illustrated in the following figure:
Figure 3: Certificate-based authentication
The client sending the logs authenticates SSB by requesting its certificate and public key. Optionally, SSB can also request a certificate from the client, thus mutual authentication is also possible.
In order to use TLS encryption in syslog-ng Premium Edition (syslog-ng PE), the following elements are required:
When using mutual authentication to verify the identity of the clients, the following elements are required:
-
A certificate must be available on the syslog-ng PE client. This certificate identifies the syslog-ng PE client.
-
The certificate of the Certificate Authority that issued the certificate of the syslog-ng PE client must be available on SSB.
Mutual authentication ensures that SSB accepts log messages only from authorized clients.
For details on configuring TLS communication in syslog-ng PE, see Configuring message sources.
Advanced Log Transfer Protocol
The syslog-ng Store Box (SSB) appli can receive log messages in a reliable way over the TCP transport layer using the Advanced Log Transfer Protocol (ALTP). ALTP is a proprietary transport protocol that prevents message loss during connection breaks. The transport protocol is used between syslog-ng Premium Edition (syslog-ng PE) hosts and SSB (for example, a client and SSB, or a client-relay-SSB), and interoperates with the flow-control and reliable disk-buffer mechanisms of syslog-ng PE, thus providing the best way to prevent message loss.
The sender detects which messages the receiver has successfully received. If messages are lost during the transfer, the sender resends the missing messages, starting from the last successfully received message. Therefore, messages are not duplicated at the receiving end in case of a connection break (however, in failover mode this is not completely ensured). ALTP also allows for connections to be encrypted.
