Chat now with support
Chat with Support

syslog-ng Premium Edition 6.0.17 - Administration Guide

Preface Chapter 1. Introduction to syslog-ng Chapter 2. The concepts of syslog-ng Chapter 3. Installing syslog-ng Chapter 4. The syslog-ng PE quick-start guide Chapter 5. The syslog-ng PE configuration file Chapter 6. Collecting log messages — sources and source drivers Chapter 7. Sending and storing log messages — destinations and destination drivers Chapter 8. Routing messages: log paths, reliability, and filters Chapter 9. Global options of syslog-ng PE Chapter 10. TLS-encrypted message transfer Chapter 11. FIPS-compliant syslog-ng Chapter 12.  Reliable Log Transfer Protocol™ Chapter 13. Reliability and minimizing the loss of log messages Chapter 14. Manipulating messages Chapter 15. Parsing and segmenting structured messages Chapter 16. Processing message content with a pattern database Chapter 17. Statistics and metrics of syslog-ng Chapter 18. Multithreading and scaling in syslog-ng PE Chapter 19. Troubleshooting syslog-ng Chapter 20. Best practices and examples

Correlating log messages

The syslog-ng PE application is able to correlate log messages identified using pattern databases.

Log messages are supposed to describe events, but applications often separate information about a single event into different log messages. For example, the Postfix e-mail server logs the sender and recipient addresses into separate log messages, or in case of an unsuccessful login attempt, the OpenSSH server sends a log message about the authentication failure, and the reason of the failure in the next message.

Of course, messages that are not so directly related can be correlated as well, for example, login-logout messages, and so on.

To correlate log messages, syslog-ng PE uses the pattern database to add messages into message-groups called contexts. A context consists of a series of log messages that are related to each other in some way, for example, the log messages of an SSH session can belong to the same context. As new messages come in, they may be added to a context. Also, when an incoming message is identified it can trigger actions to be performed, for example, generate a new message that contains all the important information that was stored previously in the context. (For details on triggering actions and generating messages, see the section called “Triggering actions for identified messages”.)

There are two attributes for pattern database rules that determine if a message matching the rule is added to a context: context-scope and context-id. The context-scope attribute acts as an early filter, selecting messages sent by the same process (${HOST}${PROGRAM}${PID} is identical), application (${HOST}${PROGRAM} is identical), or host, while the context-id actually adds the message to the context specified in the id. The context-id can be a simple string, or can contain macros or values extracted from the log messages for further filtering.

NOTE:

Message contexts are persistent and are not lost when syslog-ng PE is reloaded (SIGHUP), but are lost when syslog-ng PE is restarted.

Another parameter of a rule is the context-timeout attribute, which determines how long a context is stored, that is, how long syslog-ng PE waits for related messages to arrive. Note the following points about timeout values:

  • When a new message is added to a context, syslog-ng PE will restart the timeout using the context-timeout set for the new message.

  • When calculating if the timeout has already expired or not, syslog-ng PE uses the timestamps of the incoming messages, not system time elapsed between receiving the two messages (unless the messages do not include a timestamp, or the keep-timestamp(no) option is set). That way syslog-ng PE can be used to process and correlate already existing log messages offline. However, the timestamps of the messages must be in chronological order (that is, a new message cannot be older than the one already processed), and if a message is newer than the current system time (that is, it seems to be coming from the future), syslog-ng PE will replace its timestamp with the current system time.

    Example 16.5. How syslog-ng PE calculates context-timeout

    Consider the following two messages:

    <38>1990-01-01T14:45:25 customhostname program6[1234]: program6 testmessage
    <38>1990-01-01T14:46:25 customhostname program6[1234]: program6 testmessage

    If the context-timeout is 10 seconds and syslog-ng PE receives the messages within 1 sec, the timeout event will occour immediately, because the difference of the two timestamp (60 sec) is larger than the timeout value (10 sec).


  • Avoid using unnecessarily long timeout values on high-traffic systems, as storing the contexts for many messages can require considerable memory. For example, if two related messages usually arrive within seconds, it is not needed to set the timeout to several hours.

Example 16.6. Using message correlation

<rule xml:id="..." context-id="ssh-session" context-timeout="86400" context-scope="process">    <patterns>        <pattern>Accepted @ESTRING:usracct.authmethod: @for @ESTRING:usracct.username: @from @ESTRING:usracct.device: @port @ESTRING:: @@ANYSTRING:usracct.service@</pattern>    </patterns>...
</rule>

For details on configuring message correlation, see the description of the context-id, context-timeout, and context-scope attributes of pattern database rules.

Referencing earlier messages of the context

When using the <value> element in pattern database rules together with message correlation, you can also refer to fields and values of earlier messages of the context by adding the @<distance-of-referenced-message-from-the-current> suffix to the macro. For example, if there are three log messages in a context, and you are creating a generated message for the third log message, the ${HOST}@1 expression refers to the host field of the current (third) message in the context, the ${HOST}@2 expression refers to the host field of the previous (second) message in the context, ${PID}@3 to the PID of the first message, and so on. For example, the following message can be created from SSH login/logout messages (for details on generating new messages, see the section called “Triggering actions for identified messages”): An SSH session for ${SSH_USERNAME}@1 from ${SSH_CLIENT_ADDRESS}@2 closed. Session lasted from ${DATE}@2 to ${DATE}.

Caution:

When referencing an earlier message of the context, always enclose the field name between braces, for example, ${PID}@3. The reference will not work if you omit the braces.

NOTE:

To use a literal @ character in a template, use @@.

Example 16.7. Referencing values from an earlier message

The following action can be used to log the length of an SSH session (the time difference between a login and a logout message in the context):

<actions>    <action>        <message>            <values>                <value name="MESSAGE">An SSH session for ${SSH_USERNAME}@1 from ${SSH_CLIENT_ADDRESS}@2 closed. Session lasted from ${DATE}@2 ${DATE} </value>            </values>        </message>    </action></actions>

Triggering actions for identified messages

The syslog-ng PE application is able to generate (trigger) messages automatically if certain events occur, for example, a specific log message is received, or the correlation timeout of a message expires. Basically, you can define messages for every pattern database rule that are emitted when a message matching the rule is received. Triggering messages is often used together with message correlation, but can also be used separately.

The generated message is injected into the same place where the db-parser() statement is referenced in the log path. To post the generated message into the internal() source instead, use the inject-mode() option in the definition of the parser.

Example 16.8. Sending triggered messages to the internal() source

To send the generated messages to the internal source, use the inject-mode(internal) option:

parser p_db {db-parser(
    file("mypatterndbfile.xml")
    inject-mode(internal)
);}; 

To inject the generated messages where the pattern database is referenced, use the inject-mode(pass-through) option:

parser p_db {db-parser(
    file("mypatterndbfile.xml")
    inject-mode(pass-through)
);}; 

The generated message must be configured in the pattern database rule. It is possible to create an entire message, use macros and values extracted from the original message with pattern database, and so on.

Example 16.9. Generating messages for pattern database matches

When inserted in a pattern database rule, the following example generates a message when a message matching the rule is received.

<actions>    <action>        <message>            <values>                <value name="MESSAGE">A log message from ${HOST} matched rule number $.classifier.rule_id</value>            </values>        </message>    </action></actions>

To inherit the properties and values of the triggering message, set the inherit-properties attribute of the <message> element to TRUE. That way the triggering log message is cloned, including name-value pairs and tags. If you set any values for the message in the <action> element, they will override the values of the original message.

Example 16.10. Generating messages with inherited values

The following action generates a message that is identical to the original message, but its $PROGRAM field is set to overriding-original-program-name

<actions>    <action>        <message inherit-properties='TRUE'>            <values>                <value name="PROGRAM">overriding-original-program-name</value>            </values>        </message>    </action></actions>

For details on configuring actions, see the description of the pattern database format.

Conditional actions

To limit when a message is triggered, use the condition attribute and specify a filter expression: the action will be executed only if the condition is met. For example, the following action is executed only if the message was sent by the host called myhost.

<action condition="'${HOST}' == 'example'">

You can use the same operators in the condition that can be used in filters. For details, see the section called “Comparing macro values in filters”.

The following action can be used to log the length of an SSH session (the time difference between a login and a logout message in the context):

<actions>    <action>        <message>            <values>                <value name="MESSAGE">An SSH session for ${SSH_USERNAME}@1 from ${SSH_CLIENT_ADDRESS}@2 closed. Session lasted from ${DATE}@2 ${DATE} </value>            </values>        </message>    </action></actions>

Example 16.11. Actions based on the number of messages

The following example triggers different actions based on the number of messages in the context. This way you can check if the context contains enough messages for the event to be complete, and execute a different action if it does not.

<actions>    <action condition='"$(context-length)" >= "4"'>        <message>            <values>                <value name="PROGRAM">event</value>                <value name="MESSAGE">Event complete</value>            </values>        </message>    </action>    <action condition='"$(context-length)" < "4"'>        <message>            <values>                <value name="PROGRAM">error</value>            <value name="MESSAGE">Error detected</value>            </values>        </message>    </action></actions>

External actions

To perform an external action when a message is triggered, for example, to send the message in an e-mail, you have to route the generated messages to an external application using the program() destination.

Example 16.12. Sending triggered messages to external applications

The following sample configuration selects the triggered messages and sends them to an external script.

  1. Set a field in the triggered message that is easy to identify and filter. For example:

    <values>    <value name="MESSAGE">A log message from ${HOST} matched rule number $.classifier.rule_id</value>    <value name="TRIGGER">yes</value></values>
  2. Create a destination that will process the triggered messages.

    destination d_triggers { program("/bin/myscript"; ); };
  3. Create a filter that selects the triggered messages from the internal source.

    filter f_triggers {match("yes" value ("TRIGGER") type(string));};
  4. Create a log path that selects the triggered messages from the internal source and sends them to the script:

    log { source(s_local); filter(f_triggers); destination(d_triggers); };
  5. Create a script that will actually process the generated messages, for example:

    #!/usr/bin/perl
    while (<>) {
            # body of the script to send emails, snmp traps, and so on
    }

Actions and message correlation

Certain features of generating messages can be used only if message correlation is used as well. For details on correlating messages, see the section called “Correlating log messages”.

  • The syslog-ng PE application automatically fills the fields for the generated message based on the scope of the context, for example, the HOST and PROGRAM fields if the context-scope is program.

  • When used together with message correlation, you can also refer to fields and values of earlier messages of the context by adding the @<distance-of-referenced-message-from-the-current> suffix to the macro. For details, see the section called “Referencing earlier messages of the context”.

    Example 16.13. Referencing values from an earlier message

    The following action can be used to log the length of an SSH session (the time difference between a login and a logout message in the context):

    <actions>    <action>        <message>            <values>                <value name="MESSAGE">An SSH session for ${SSH_USERNAME}@1 from ${SSH_CLIENT_ADDRESS}@2 closed. Session lasted from ${DATE}@2 ${DATE} </value>            </values>        </message>    </action></actions>

  • You can use the name-value pairs of other messages of the context. If you set the inherit-properties attribute of the generated message to context, syslog-ng PE collects every name-value pair from each message stored in the context, and includes them in the generated message. This means that you can refer to a name-value pair without having to know which message of the context included it. If a name-value pair appears in multiple messages of the context, the value in the latest message will be used. To refer to an earlier value, use the @<distance-of-referenced-message-from-the-current> suffix format.

    <action>    <message inherit-properties='context'>

    Example 16.14. Using the inherit-properties option

    For example, if inherit-properties is set to context, and you have a rule that collects SSH login and logout messages to the same context, you can use the following value to generate a message collecting the most important information form both messages, including the beginning and end date.

    <value name="MESSAGE">An SSH session for ${SSH_USERNAME} from ${SSH_CLIENT_ADDRESS} closed. Session lasted from ${DATE}@2 to $DATE pid: $PID.</value>

    The following is a detailed rule for this purpose.

    <patterndb version='4' pub_date='2015-04-13'>    <ruleset name='sshd' id='12345678'>        <pattern>sshd</pattern>            <rules>                <!-- The pattern database rule for the first log message -->                <rule provider='me' id='12347598' class='system'
                        context-id="ssh-login-logout" context-timeout="86400"
                        context-scope="process">                <!-- Note the context-id that groups together the
                    relevant messages, and the context-timeout value that
                    determines how long a new message can be added to the
                    context  -->                    <patterns>                        <pattern>Accepted @ESTRING:SSH.AUTH_METHOD: @for @ESTRING:SSH_USERNAME: @from @ESTRING:SSH_CLIENT_ADDRESS: @port @ESTRING:: @@ANYSTRING:SSH_SERVICE@</pattern>                        <!-- This is the actual pattern used to identify
                            the log message. The segments between the @
                            characters are parsers that recognize the variable
                            parts of the message - they can also be used as
                            macros.  -->                    </patterns>                </rule>                <!-- The pattern database rule for the fourth log message -->                <rule provider='me' id='12347599' class='system' context-id="ssh-login-logout" context-scope="process">                    <patterns>                         <pattern>pam_unix(sshd:session): session closed for user @ANYSTRING:SSH_USERNAME@</pattern>                    </patterns>                    <actions>                        <action>                            <message inherit-properties='context'>                                <values>                                    <value name="MESSAGE">An SSH session for ${SSH_USERNAME} from ${SSH_CLIENT_ADDRESS} closed. Session lasted from ${DATE}@2 to $DATE pid: $PID.</value>                                    <value name="TRIGGER">yes</value>                                    <!-- This is the new log message
                                        that is generated when the logout
                                        message is received. The macros ending
                                        with @2 reference values of the
                                        previous message from the context. -->                                </values>                            </message>                        </action>                    </actions>                </rule>            </rules>    </ruleset></patterndb>

  • It is possible to generate a message when the context-timeout of the original message expires and no new message is added to the context during this time. To accomplish this, include the trigger="timeout" attribute in the action element:

    <action trigger="timeout">

    Example 16.15. Sending alert when a client disappears

    The following example shows how to combine various features of syslog-ng PE to send an e-mail alert if a client stops sending messages.

    • Configure your clients to send MARK messages periodically. It is enough to configure MARK messages for the destination that forwards your log messages to your syslog-ng PE server (mark-mode(periodical)).

    • On your syslog-ng PE server, create a pattern database rule that matches on the incoming MARK messages. In the rule, set the context-scope attribute to host, and the context-timeout attribute to a value that is higher than the mark-freq value set on your clients (by default, mark-freq is 1200 seconds, so set context-timeout at least to 1500 seconds, but you might want to use a higher value, depending on your environment).

    • Add an action to this rule that sends you an e-mail alert if the context-timeout expires, and the server does not receive a new MARK message (<action trigger="timeout">).

    • On your syslog-ng PE server, use the pattern database in the log path that handles incoming log messages.


Creating pattern databases

Using pattern parsers

Pattern parsers attempt to parse a part of the message using rules specific to the type of the parser. Parsers are enclosed between @ characters. The syntax of parsers is the following:

  • a beginning @ character,

  • the type of the parser written in capitals,

  • optionally a name,

  • parameters of the parser, if any, and

  • a closing @ character.

Example 16.16. Pattern parser syntax

A simple parser:

@STRING@

A named parser:

@STRING:myparser_name@

A named parser with a parameter:

@STRING:myparser_name:*@

A parser with a parameter, but without a name:

@STRING::*@

Patterns and literals can be mixed together. For example, to parse a message that begins with the Host: string followed by an IP address (for example, Host: 192.168.1.1), the following pattern can be used: Host:@IPv4@.

NOTE:

Note that using parsers is a CPU-intensive operation. Use the ESTRING and QSTRING parsers whenever possible, as these can be processed much faster than the other parsers.

Example 16.17. Using the STRING and ESTRING parsers

For example, if the message is user=joe96 group=somegroup, @STRING:mytext:@ parses only to the first non-alphanumeric character (=), parsing only user. @STRING:mytext:=@ parses the equation mark as well, and proceeds to the next non-alphanumeric character (the whitespace), resulting in user=joe96 being parsed. @STRING:mytext:= @ will parse the whitespace as well, and proceed to the next non-alphanumeric non-equation mark non-whitespace character, resulting in user=joe96 group=somegroup.

Of course, usually it is better to parse the different values separately, like this: "user=@STRING:user@ group=@STRING:group@".

If the username or the group may contain non-alphanumeric characters, you can either include these in the second parameter of the parser (as shown at the beginning of this example), or use an ESTRING parser to parse the message till the next whitespace: "user=@ESTRING:user: @group=@ESTRING:group: @".


Pattern parsers of syslog-ng PE

The following parsers are available in syslog-ng PE.

@ANYSTRING@

Parses everything to the end of the message, you can use it to collect everything that is not parsed specifically to a single macro. In that sense its behavior is similar to the greedy() option of the CSV parser.

@DOUBLE@

An obsolete alias of the @FLOAT@ parser.

@ESTRING@

This parser has a required parameter that acts as the stopcharacter: the parser parses everything until it finds the stopcharacter. For example, to stop by the next " (double quote) character, use @ESTRING::"@. You can use the colon (:) as stopcharacter as well, for example: @ESTRING:::@. You can also specify a stopstring instead of a single character, for example, @ESTRING::stop_here.@. The @ character cannot be a stopcharacter, nor can line-breaks or tabs.

@FLOAT@

A floating-point number that may contain a dot (.) character. (Up to syslog-ng 3.1, the name of this parser was @DOUBLE@.)

@IPv4@

Parses an IPv4 IP address (numbers separated with a maximum of 3 dots).

@IPv6@

Parses any valid IPv6 IP address.

@IPvANY@

Parses any IP address.

@NUMBER@

A sequence of decimal (0-9) numbers (for example, 1, 0687, and so on). Note that if the number starts with the 0x characters, it is parsed as a hexadecimal number, but only if at least one valid character follows 0x. A leading hyphen () is accepted for non-hexadecimal numbers, but other separator characters (for example, dot or comma) are not. To parse floating-point numbers, use the @FLOAT@ parser.

@QSTRING@

Parse a string between the quote characters specified as parameter. Note that the quote character can be different at the beginning and the end of the quote, for example: @QSTRING::"@ parses everything between two quotation marks ("), while @QSTRING:&lt;&gt;@ parses from an opening bracket to the closing bracket. The @ character cannot be a quote character, nor can line-breaks or tabs.

@STRING@

A sequence of alphanumeric characters (0-9, A-z), not including any whitespace. Optionally, other accepted characters can be listed as parameters (for example, to parse a complete sentence, add the whitespace as parameter, like: @STRING:: @). Note that the @ character cannot be a parameter, nor can line-breaks or tabs.

The syslog-ng pattern database format

Pattern databases are XML files that contain rules describing the message patterns. For sample pattern databases, see the section called “Downloading sample pattern databases”.

The following scheme describes the V4 format of the pattern database. This format is used by syslog-ng PE4 F1 and later, and is backwards-compatible with the earlier V3 format.

For a sample database containing only a single pattern, see Example 16.18, “A V4 pattern database containing a single rule”.

TIP:

Use the pdbtool utility that is bundled with syslog-ng to test message patterns and convert existing databases to the latest format. For details, see pdbtool(1).

To automatically create an initial pattern database from an existing log file, use the pdbtool patternize command. For details, see the section called “The patternize command”.

Example 16.18. A V4 pattern database containing a single rule

The following pattern database contains a single rule that matches a log message of the ssh application. A sample log message looks like:

Accepted password for sampleuser from 10.50.0.247 port 42156 ssh2

The following is a simple pattern database containing a matching rule.

<patterndb version='4' pub_date='2010-10-17'>    <ruleset name='ssh' id='123456678'>        <pattern>ssh</pattern>            <rules>                <rule provider='me' id='182437592347598' class='system'>                    <patterns>                        <pattern>Accepted @QSTRING:SSH.AUTH_METHOD: @ for@QSTRING:SSH_USERNAME: @from\ @QSTRING:SSH_CLIENT_ADDRESS: @port @NUMBER:SSH_PORT_NUMBER:@ ssh2</pattern>                    </patterns>                </rule>            </rules>    </ruleset></patterndb>

Note that the rule uses macros that refer to parts of the message, for example, you can use the ${SSH_USERNAME} macro refer to the username used in the connection.

The following is the same example, but with a test message and test values for the parsers.

<patterndb version='4' pub_date='2010-10-17'>    <ruleset name='ssh' id='123456678'>        <pattern>ssh</pattern>            <rules>                <rule provider='me' id='182437592347598' class='system'>                    <patterns>                        <pattern>Accepted @QSTRING:SSH.AUTH_METHOD: @ for@QSTRING:SSH_USERNAME: @from\ @QSTRING:SSH_CLIENT_ADDRESS: @port @NUMBER:SSH_PORT_NUMBER:@ ssh2</pattern>                    </patterns>                    <examples>                        <example>                            <test_message>Accepted password for sampleuser from 10.50.0.247 port 42156 ssh2</test_message>                            <test_values>                                <test_value name="SSH.AUTH_METHOD">password</test_value>                                <test_value name="SSH_USERNAME">sampleuser</test_value>                                <test_value name="SSH_CLIENT_ADDRESS">10.50.0.247</test_value>                                <test_value name="SSH_PORT_NUMBER">42156</test_value>                            </test_values>                       </example>                    </examples>                </rule>            </rules>    </ruleset></patterndb>

Element: patterndb

Location

/patterndb

Description

The container element of the pattern database.

Attributes
  • version: The schema version of the pattern database. The current version is 4.

  • pubdate: The publication date of the XML file.

Children
Example
<patterndb version='4' pub_date='2010-10-25'>

Element: ruleset

Location

/patterndb/ruleset

Description

A container element to group log patterns for an application or program. A <patterndb> element may contain any number of <ruleset> elements.

Attributes
  • name: The name of the application. Note that the function of this attribute is to make the database more readable, syslog-ng uses the <pattern> element to identify the applications sending log messages.

  • id: A unique ID of the application, for example, the md5 sum of the name attribute.

  • description: OPTIONAL — A description of the ruleset or the application.

  • url: OPTIONAL — An URL referring to further information about the ruleset or the application.

Example
<ruleset name='su' id='480de478-d4a6-4a7f-bea4-0c0245d361e1'>

Element: patterns

Location

/patterndb/ruleset/patterns

Description

A container element. A <patterns> element may contain any number of <pattern> elements.

Attributes

N/A

Children
  • pattern: The name of the application — syslog-ng matches this value to the ${PROGRAM} header of the syslog message to find the rulesets applicable to the syslog message.

    Specifying multiple patterns is useful if two or more applications have different names (that is, different ${PROGRAM} fields), but otherwise send identical log messages.

    It is not necessary to use multiple patterns if only the end of the ${PROGRAM} fields is different, use only the beginning of the ${PROGRAM} field as the pattern. For example, the Postfix e-mail server sends messages using different process names, but all of them begin with the postfix string.

    NOTE:

    If the <pattern> element of a ruleset is not specified, syslog-ng PE will use this ruleset as a fallback ruleset: it will apply the ruleset to messages that have an empty PROGRAM header, or if none of the program patterns matched the PROGRAM header of the incoming message.

Example
<patterns>    <pattern>firstapplication</pattern>    <pattern>otherapplication</pattern></patterns>

Element: rules

Location

/patterndb/ruleset/rules

Description

A container element for the rules of the ruleset.

Attributes

N/A

Children
Example
<rules>    <rule provider='me' id='182437592347598' class='system'>        <patterns>            <pattern>Accepted @QSTRING:SSH.AUTH_METHOD: @ for@QSTRING:SSH_USERNAME: @from\ @QSTRING:SSH_CLIENT_ADDRESS: @port @NUMBER:SSH_PORT_NUMBER:@ ssh2</pattern>        </patterns>    </rule></rules>

Element: rule

Location

/patterndb/ruleset/rules/rule

Description

An element containing message patterns and how a message that matches these patterns is classified.

NOTE:

If the following characters appear in the message, they must be escaped in the rule as follows:

  • @: Use @@, for example user@@example.com

  • <: Use &lt;

  • >: Use &gt;

  • &: Use &amp;

The <rules> element may contain any number of <rule> elements.

Attributes
  • provider: The provider of the rule. This is used to distinguish between who supplied the rule, that is, if it has been created by One Identity, or added to the XML by a local user.

  • id: The globally unique ID of the rule.

  • class: The class of the rule — syslog-ng assigns this class to the messages matching a pattern of this rule.

  • context-id: OPTIONAL — An identifier to group related log messages when using the pattern database to correlate events. The ID can be a descriptive string describing the events related to the log message (for example, ssh-sessions for log messages related to SSH traffic), but can also contain macros to generate IDs dynamically. When using macros in IDs, see also the context-scope attribute. For details on correlating messages, see the section called “Correlating log messages”.

    NOTE:

    The syslog-ng PE application determines the context of the message after the pattern matching is completed. This means that macros and name-value pairs created by the matching pattern database rule can be used as context-id macros.

  • context-timeout: OPTIONAL — The number of seconds the context is stored. Note that for high-traffic log servers, storing open contexts for long time can require significant amount of memory. For details on correlating messages, see the section called “Correlating log messages”.

  • context-scope: OPTIONAL — Specifies which messages belong to the same context. This attribute is used to determine the context of the message if the context-id does not specify any macros. Usually, context-scope acts a filter for the context, with context-id refining the filtering if needed. The context-scope attribute has the following possible values:

    • process: Only messages that are generated by the same process of a client belong to the same context, that is, messages that have identical ${HOST}, ${PROGRAM} and ${PID} values. This is the default behavior of syslog-ng PE if context-scope is not specified.

    • program: Messages that are generated by the same application of a client belong to the same context, that is, messages that have identical ${HOST} and ${PROGRAM} values.

    • host: Every message generated by a client belongs to the same context, only the ${HOST} value of the messages must be identical.

    • global: Every message belongs to the same context.

    NOTE:

    Using the context-scope attribute is significantly faster than using macros in the context-id attribute.

    For details on correlating messages, see the section called “Correlating log messages”.

Children
Example
<rule provider='balabit' id='f57196aa-75fd-11dd-9bba-001e6806451b' class='violation'>

The following example specifies attributes for correlating messages as well. For details on correlating messages, see the section called “Correlating log messages”.

<rule provider='balabit' id='f57196aa-75fd-11dd-9bba-001e6806451b' class='violation' context-id='same-session' context-scope='process' context-timeout='360'>

Element: patterns

Location

/patterndb/ruleset/rules/rule/patterns

Description

An element containing the patterns of the rule. If a <patterns> element contains multiple <pattern> elements, the class of the <rule> is assigned to every syslog message matching any of the patterns.

Attributes

N/A

Children
  • pattern: A pattern describing a log message. This element is also called message pattern. For example:

    <pattern>+ ??? root-</pattern>

    NOTE:

    Support for XML entities is limited, you can use only the following entities: &amp; &lt; &gt; &quot; &apos;. User-defined entities are not supported.

  • description: OPTIONAL — A description of the pattern or the log message matching the pattern.

  • urls

  • values

  • examples

Example
<patterns>    <pattern>Accepted @QSTRING:SSH.AUTH_METHOD: @ for@QSTRING:SSH_USERNAME: @from\ @QSTRING:SSH_CLIENT_ADDRESS: @port @NUMBER:SSH_PORT_NUMBER:@ ssh2</pattern></patterns>

Element: urls

Description

OPTIONAL — An element containing one or more URLs referring to further information about the patterns or the matching log messages.

Attributes

N/A

Children
  • url: OPTIONAL — An URL referring to further information about the patterns or the matching log messages.

Example

N/A

Element: values

Location

/patterndb/ruleset/rules/rule/patterns/values

Description

OPTIONAL — Name-value pairs that are assigned to messages matching the patterns, for example, the representation of the event in the message according to the Common Event Format (CEF) or Common Event Exchange (CEE). The names can be used as macros to reference the assigned values.

Attributes

N/A

Children
Example
<values>    <value name=".classifier.outcome">/Success</value></values>

Element: examples

Location

/patterndb/ruleset/rules/rule/patterns/examples

Description

OPTIONAL — A container element for sample log messages that should be recognized by the pattern. These messages can be used also to test the patterns and the parsers.

Attributes

N/A

Children
Example
<examples>    <example>        <test_message>Accepted password for sampleuser from 10.50.0.247 port 42156 ssh2</test_message>        <test_values>            <test_value name="SSH.AUTH_METHOD">password</test_value>            <test_value name="SSH_USERNAME">sampleuser</test_value>            <test_value name="SSH_CLIENT_ADDRESS">10.50.0.247</test_value>            <test_value name="SSH_PORT_NUMBER">42156</test_value>        </test_values>    </example></examples>

Element: example

Description

OPTIONAL — A container element for a sample log message.

Attributes

N/A

Children
  • test_message: OPTIONAL — A sample log message that should match this pattern. For example:

    <test_message program="myapplication">Content filter has been enabled</test_message>
    • program: The program pattern of the test message. For example:

      <test_message program="proftpd">ubuntu (::ffff:192.168.2.179[::ffff:192.168.2.179]) - FTP session closed.</test_message>
  • test_values: OPTIONAL — A container element to test the results of the parsers used in the pattern.

    • test_value: OPTIONAL — The expected value of the parser when matching the pattern to the test message. For example:

      <test_value name=".dict.ContentFilter">enabled</test_value>
      • name: The name of the parser to test.

Example
<examples>    <example>        <test_message>Accepted password for sampleuser from 10.50.0.247 port 42156 ssh2</test_message>        <test_values>            <test_value name="SSH.AUTH_METHOD">password</test_value>            <test_value name="SSH_USERNAME">sampleuser</test_value>            <test_value name="SSH_CLIENT_ADDRESS">10.50.0.247</test_value>            <test_value name="SSH_PORT_NUMBER">42156</test_value>        </test_values>    </example></examples>

Element: actions

Location

/patterndb/ruleset/actions

Description

OPTIONAL — A container element for actions that are performed if a message is recognized by the pattern. For details on actions, see the section called “Triggering actions for identified messages”.

Attributes

N/A

Children
Example

Example 16.19. Generating messages for pattern database matches

When inserted in a pattern database rule, the following example generates a message when a message matching the rule is received.

<actions>    <action>        <message>            <values>                <value name="MESSAGE">A log message from ${HOST} matched rule number $.classifier.rule_id</value>            </values>        </message>    </action></actions>

To inherit the properties and values of the triggering message, set the inherit-properties attribute of the <message> element to TRUE. That way the triggering log message is cloned, including name-value pairs and tags. If you set any values for the message in the <action> element, they will override the values of the original message.

Example 16.20. Generating messages with inherited values

The following action generates a message that is identical to the original message, but its $PROGRAM field is set to overriding-original-program-name

<actions>    <action>        <message inherit-properties='TRUE'>            <values>                <value name="PROGRAM">overriding-original-program-name</value>            </values>        </message>    </action></actions>

Element: action

Location

/patterndb/ruleset/actions/action

Description

OPTIONAL — A container element describing an action that is performed when a message matching the rule is received.

Attributes
  • condition: A syslog-ng filter expression. The action is performed only if the message matches the filter. The filter can include macros and name-value pairs extracted from the message.

  • rate: Specifies maximum how many messages should be generated in the specified time period in the following format: <number-of-messages>/<period-in-seconds>. For example: 1/60 allows 1 message per minute. Rates apply within the scope of the context, that is, if context-scope="host" and rate="1/60", then maximum one message is generated per minute for every host that sends a log message matching the rule. Excess messages are dropped. Note that when applying the rate to the generated messages, syslog-ng PE uses the timestamps of the log messages, similarly to calculating the context-timeout. That way rate is applied correctly even if the log messages are processed offline.

  • trigger: Specifies when the action is executed. The trigger attribute has the following possible values:

    • match: Execute the action immediately when a message matching the rule is received.

    • timeout: Execute the action when the correlation timer (context-timeout) expires. This is available only if actions are used together with correlating messages.

Children
  • message: A container element storing the message to be sent when the action is executed. Currently syslog-ng PE sends these messages to the internal() destination.

  • values: A container element for values and fields that are used to create the message generated by the action.

    • value: Sets the value of the message field specified in the name attribute of the element. For example, to specify the body of the generated message, use the following syntax:

      <value name="MESSAGE">A log message matched rule number $.classifier.rule_id</value>

      Note that currently it is not possible to add DATE, FACILITY, or SEVERITY fields to the message.

      When the action is used together with message correlation, the syslog-ng PE application automatically adds fields to the message based on the context-scope parameter. For example, using context-scope="process" automatically fills the HOST, PROGRAM, and PID fields of the generated message.

    • name: Name of the message field set by the value element.

Example

Example 16.21. Generating messages for pattern database matches

When inserted in a pattern database rule, the following example generates a message when a message matching the rule is received.

<actions>    <action>        <message>            <values>                <value name="MESSAGE">A log message from ${HOST} matched rule number $.classifier.rule_id</value>            </values>        </message>    </action></actions>

To inherit the properties and values of the triggering message, set the inherit-properties attribute of the <message> element to TRUE. That way the triggering log message is cloned, including name-value pairs and tags. If you set any values for the message in the <action> element, they will override the values of the original message.

Example 16.22. Generating messages with inherited values

The following action generates a message that is identical to the original message, but its $PROGRAM field is set to overriding-original-program-name

<actions>    <action>        <message inherit-properties='TRUE'>            <values>                <value name="PROGRAM">overriding-original-program-name</value>            </values>        </message>    </action></actions>

Element: tags

Location

/patterndb/ruleset/tags

Description

OPTIONAL — An element containing custom keywords (tags) about the messages matching the patterns. The tags can be used to label specific events (for example user logons). It is also possible to filter on these tags later (for details, see the section called “Tagging messages”). Starting with syslog-ng Premium Edition 3.2, the list of tags assigned to a message can be referenced with the ${TAGS} macro.

Attributes

N/A

Children
  • tag: OPTIONAL — A keyword or tags applied to messages matching the rule.

Example
<tags><tag>UserLogin</tag></tags>

Chapter 17. Statistics and metrics of syslog-ng

Periodically, syslog-ng sends a message containing statistics about the received messages, and about any lost messages since the last such message. It includes a processed entry for every source and destination, listing the number of messages received or sent, and a dropped entry including the IP address of the server for every destination where syslog-ng has lost messages. The center(received) entry shows the total number of messages received from every configured sources.

The following is a sample log statistics message for a configuration that has a single source (s_local) and a network and a local file destination (d_network and d_local, respectively). All incoming messages are sent to both destinations.

Log statistics;
        dropped='tcp(AF_INET(192.168.10.1:514))=6439',
        processed='center(received)=234413',
        processed='destination(d_tcp)=234413',
        processed='destination(d_local)=234413',
        processed='source(s_local)=234413'

Log statistics can be also retrieved on-demand using one of the following options:

  • Using the syslog-ng-ctl stats command.

  • Using the syslog-ng-query sum "*" command. Note that the output of the syslog-ng-query command is better structured, and also allows you to select which metrics you want to display. For details, see syslog-ng-query(1).

  • Use the socat application: echo STATS | socat -vv UNIX-CONNECT:/opt/syslog-ng/var/run/syslog-ng.ctl -

  • If you have an OpenBSD-style netcat application installed, use the echo STATS | nc -U /opt/syslog-ng/var/run/syslog-ng.ctl command. Note that the netcat included in most Linux distributions is a GNU-style version that is not suitable to query the statistics of syslog-ng.

The statistics include a list of source groups and destinations, as well as the number of processed messages for each. The verbosity of the statistics can be set using the stats-level() option. For details, see the section called “Global options”. An example output is shown below.

src.internal;s_all#0;;a;processed;6445
src.internal;s_all#0;;a;stamp;1268989330
destination;df_auth;;a;processed;404
destination;df_news_dot_notice;;a;processed;0
destination;df_news_dot_err;;a;processed;0
destination;d_ssb;;a;processed;7128
destination;df_uucp;;a;processed;0
source;s_all;;a;processed;7128
destination;df_mail;;a;processed;0
destination;df_user;;a;processed;1
destination;df_daemon;;a;processed;1
destination;df_debug;;a;processed;15
destination;df_messages;;a;processed;54
destination;dp_xconsole;;a;processed;671
dst.tcp;d_network#0;10.50.0.111:514;a;dropped;5080
dst.tcp;d_network#0;10.50.0.111:514;a;processed;7128
dst.tcp;d_network#0;10.50.0.111:514;a;stored;2048
destination;df_syslog;;a;processed;6724
destination;df_facility_dot_warn;;a;processed;0
destination;df_news_dot_crit;;a;processed;0
destination;df_lpr;;a;processed;0
destination;du_all;;a;processed;0
destination;df_facility_dot_info;;a;processed;0
center;;received;a;processed;0
destination;df_kern;;a;processed;70
center;;queued;a;processed;0
destination;df_facility_dot_err;;a;processed;0

The statistics are semicolon separated: every line contains statistics for a particular object (for example source, destination, tag, and so on). The statistics have the following fields:

  1. The type of the object (for example dst.file, tag, src.facility)

  2. The ID of the object used in the syslog-ng configuration file, for example d_internal or source.src_tcp. The #0 part means that this is the first destination in the destination group.

  3. The instance ID (destination) of the object, for example the filename of a file destination, or the name of the application for a program source or destination.

  4. The status of the object. One of the following:

    • a - active. At the time of quering the statistics, the source or the destination was still alive (it continuously received statistical data).

    • d - dynamic. Such objects may not be continuously available, for example, like statistics based on the sender's hostname.

    • o - This object was once active, but stopped receiving messages. (For example a dynamic object may disappear and become orphan.)

      NOTE:

      The syslog-ng PE application stores the statistics of the objects when syslog-ng PE is reloaded. However, if the configuration of syslog-ng PE was changed since the last reload, the statistics of orphaned objects are deleted.

  5. The type of the statistics:

    • processed: The number of messages that successfully reached their destination driver. Note that this does not necessarily mean that the destination driver successfully delivered the messages (for example, written to disk or sent to a remote server).

    • dropped: The number of dropped messages — syslog-ng PE could not send the messages to the destination and the output buffer got full, so messages were dropped by the destination driver.

    • stored: The number of messages stored in the message queue of the destination driver, waiting to be sent to the destination.

    • suppressed: The number of suppressed messages (if the suppress() feature is enabled).

    • stamp: The UNIX timestamp of the last message sent to the destination.

  6. The number of such messages.

NOTE:

Certain statistics are available only if the stats-level() option is set to a higher value.

When receiving messages with non-standard facility values (that is, higher than 23), these messages will be listed as other facility instead of their facility number.

To reset the statistics to zero, use the following command: syslog-ng-ctl stats --reset

Related Documents