The concepts of syslog-ng
This chapter discusses the technical concepts of syslog-ng.
The philosophy of syslog-ng
Typically, syslog-ng is used to manage log messages and implement centralized logging, where the aim is to collect the log messages of several devices on a single, central log server. The different devices — called syslog-ng clients — all run syslog-ng, and collect the log messages from the various applications, files, and other sources. The clients send all important log messages to the remote syslog-ng server, which sorts and stores them.
Logging with syslog-ng
The syslog-ng application reads incoming messages and forwards them to the selected destinations. The syslog-ng application can receive messages from files, remote hosts, and other sources.
Log messages enter syslog-ng in one of the defined sources, and are sent to one or more destinations.
Sources and destinations are independent objects, log paths define what syslog-ng does with a message, connecting the sources to the destinations. A log path consists of one or more sources and one or more destinations: messages arriving from a source are sent to every destination listed in the log path. A log path defined in syslog-ng is called a log statement.
Optionally, log paths can include filters. Filters are rules that select only certain messages, for example, selecting only messages sent by a specific application. If a log path includes filters, syslog-ng sends only the messages satisfying the filter rules to the destinations set in the log path.
Other optional elements that can appear in log statements are parsers and rewriting rules. Parsers segment messages into different fields to help processing the messages, while rewrite rules modify the messages by adding, replacing, or removing parts of the messages.
The route of a log message in syslog-ng
Purpose:
The following procedure illustrates the route of a log message from its source on the syslog-ng client to its final destination on the central syslog-ng server.
Figure 1: The route of a log message
Steps:
-
A device or application sends a log message to a source on the syslog-ng client. For example, an Apache web server running on Linux enters a message into the /var/log/apache file.
-
The syslog-ng client running on the web server reads the message from its /var/log/apache source.
-
The syslog-ng client processes the first log statement that includes the /var/log/apache source.
-
The syslog-ng client performs optional operations (message filtering, parsing, and rewriting) on the message, for example, it compares the message to the filters of the log statement (if any). If the message complies with all filter rules, syslog-ng sends the message to the destinations set in the log statement, for example, to the remote syslog-ng server.
Caution: Message filtering, parsing, and rewriting is performed in the order that the operations appear in the log statement.
NOTE: The syslog-ng client sends a message to all matching destinations by default. As a result, a message may be sent to a destination more than once, if the destination is used in multiple log statements. To prevent such situations, use the final flag in the destination statements. For details, see Log statement flags.
-
The syslog-ng client processes the next log statement that includes the /var/log/apache source, repeating Steps 3-4.
-
The message sent by the syslog-ng client arrives from a source set in the syslog-ng server.
-
The syslog-ng server reads the message from its source and processes the first log statement that includes that source.
-
The syslog-ng server performs optional operations (message filtering, parsing, and rewriting) on the message, for example, it compares the message to the filters of the log statement (if any). If the message complies with all filter rules, syslog-ng sends the message to the destinations set in the log statement.
Caution: Message filtering, parsing, and rewriting is performed in the order that the operations appear in the log statement.
-
The syslog-ng server processes the next log statement, repeating Steps 7-9.
NOTE: The syslog-ng application can stop reading messages from its sources if the destinations cannot process the sent messages. This feature is called flow-control and is detailed in Managing incoming and outgoing messages with flow-control.
