Chat now with support
Chat with Support

syslog-ng Store Box 7.4.0 - Administration Guide

Preface Introduction The concepts of SSB The Welcome Wizard and the first login Basic settings User management and access control Managing SSB Configuring message sources Storing messages on SSB Forwarding messages from SSB Log paths: routing and processing messages Configuring syslog-ng options Searching log messages Searching the internal messages of SSB Classifying messages with pattern databases The SSB RPC API Monitoring SSB Troubleshooting SSB Security checklist for configuring SSB Glossary

Parsing key-value pairs

The syslog-ng Store Box(SSB) appliance can separate a message consisting of key-value pairs (for example, Postfix log messages) into name-value pairs. The parsed values are automatically added to the metadata about the message, and you can display them on the SSB search interface as dynamic columns. You can specify the separator character to parse different log messages, for example, colon (:) to parse MySQL log messages, or the equal sign (=) for firewall logs. For details on when the key-value parser is executed related to other message processing operations, see the following list.

Caution:

If the names of keys in the message is the same as the names of SSB soft macros, the value from the parsed message will overwrite the value of the macro. For example, the PROGRAM=value1, MESSAGE=value2 content will overwrite the ${PROGRAM} and ${MESSAGE} macros. To avoid overwriting such macros, use the prefix() option.

Hard macros cannot be modified, so they will not be overwritten. For details on the macro types, see .

The parser discards message sections that are not key=value pairs, even if they appear between key=value pairs that can be parsed.

  1. Parse the message as a syslog message (unless message parsing is explicitly disabled for the source).

  2. Classify the message using a pattern database.

  3. Modify the message using rewrite rules (before filtering).

  4. Filter the messages, for example, based on sender hostname or message content. If the message does not match the configured filter, syslog-ng Store Box(SSB) will not send it to the destination.

  5. Parse the text of the message (that is, the ${MESSAGE} part) using a key-value parser or the sudo parser.

  6. Modify the message using rewrite rules (after filtering and other parsing).

  7. SSB sends the message to the destinations set in the log path. The destinations are local, optionally encrypted files on SSB, or remote servers, such as a database server.

NOTE: If a log message contains the same key multiple times (for example, key1=value1, key2=value2, key1=value3, key3=value4, key1=value5), then SSB stores only the last (rightmost) value for the key. Using the previous example, SSB will store the following pairs: key1=value5, key2=value2, key3=value4..

NOTE: The names of the keys can contain only the following characters: numbers (0-9), letters (a-z,A-Z), underscore (_), dot (.), hyphen (-). Other special characters are not permitted.

To configure parsing key-value pairs

  1. Navigate to Log > Parsers and select . A new parser is added to the list of parsers.

    Figure 202: Log > Parsers — Creating a key=value parser

  2. Enter a name for the parser.

  3. Enter the character that separates the keys from the values in the incoming messages into the Value separator character field. For example, if your messages look like key1:value1, key2:value2, key3:value3, enter :.

  4. Enter the character or string that separates the key-value pairs from each other into the Pair separator string field. For example, if your messages look like key1:value1, key2:value2, key3:value3, enter ,. If you use a string, it can be a maximum of 20 characters long.
  5. Enter a prefix before the key part of the parsed key-value pairs to help further processing into the Namespace field. For example, to insert the my-parsed-data prefix, enter my-parsed-data.

    NOTE: Consider the following when configuring a prefix:

    • SSB automatically adds the .SDATA. prefix before the value you enter into the Namespace field. That way these values are automatically included in the structured data (SDATA) part of the log message if you forward the message using the IETF-syslog protocol.

    • SSB automatically adds a dot (.) character as a separator between the namespace and the key parsed from the message.

  6. Click .

  7. Navigate to Log > Paths.

  8. Select the path where you want to use the parser.

  9. In the Parser field, Custom group, select the parser you want to use in this log path.

    Figure 203: Log > Paths — Using a key=value parser in the log path

  10. Click .

Configuring syslog-ng options

There are several options of the syslog-ng server running on syslog-ng Store Box(SSB) that can be configured. These include:

General syslog-ng settings

To configure the general options of the syslog-ng server running on the syslog-ng Store Box (SSB) appliance, navigate to Log > Options. The following options are available.

NOTE: Options related to name resolution are described in Using name resolution on SSB.

Figure 204: Log > Options — Configuring syslog-ng options

  • Message size: Specifies the maximum length of incoming log messages in bytes. This option corresponds to the log-msg-size() parameter of syslog-ng. The maximum value of this parameter is 1000000 (1 MB).

    NOTE: To be able to edit the Message size, you must have write/perform permission for the Basic Settings > System page. For details on how to assign user rights, see Managing user rights and usergroups.

  • Wait time between polls: The time to wait in milliseconds before checking if new messages have arrived to a source. This option corresponds to the time-sleep() parameter of syslog-ng.

  • Idle time before destination is closed: The time to wait in seconds before an idle destination file is closed. This option corresponds to the time-reap() parameter of syslog-ng.

  • Cipher: Select the cipher method used to encrypt the logstore. The following cipher methods are available: aes-128-cbc, aes-128-cfb, aes-128-cfb1, aes-128-cfb8, aes-128-ecb, aes-128-ofb, aes-192-cbc, aes-192-cfb, aes-192-cfb1, aes-192-cfb8, aes-192-ecb, aes-192-ofb, aes-256-cbc, aes-256-cfb, aes-256-cfb1, aes-256-cfb8, aes-256-ecb, aes-256-ofb, aes128, aes192, aes256, bf, bf-cbc, bf-cfb, bf-ecb, bf-ofb, cast5-cbc, cast5-cfb, des-cbc, des-cfb, des-cfb1, des-cfb8, des-ecb, des-ede, des-ede-cbc, des-ede-cfb, des-ede-ofb, des-ede3, des-ede3-cbc, des-ede3-cfb, des-ede3-ofb, des-ofb, desx-cbc, rc2-40-cbc, rc2-64-cbc, rc2-cbc, rc2-cfb, rc2-ecb, rc2-ofb, rc4, and rc4-40.

    By default, SSB uses the aes-256-cbc method.

  • Digest: Select the digest method to use. The following digest methods are available: MD4, MD5, SHA-1, RIPEMD-160, SHA-224, SHA-256, SHA-384, and SHA-512.

    By default, SSB uses the SHA-256 method.

    Caution:

    The size of the digest hash must be equal to or larger than the key size of the cipher method. For example, to use the aes-256-cbc cipher method, the digest method must be at least SHA-256.

Time stamping configuration on SSB

To configure the time stamping options of syslog-ng Store Box (SSB), navigate to Log > Options. The following options are available:

  • Time stamp server: Select the time stamping server to use for signing encrypted logspaces. To use the built-in time stamp server of SSB, select Local.

    To use an external time stamping server, select Remote and enter the address of the server into the Server URL field in the following format:

    http://<IP address>:<port number>/

    For example:

    http://10.50.50.50:8080/

    Note that currently only plain HTTP services are supported, password-protected and HTTPS services are not supported.

    Caution:

    SSB currently supports only time stamping servers that use the Internet X.509 Public Key Infrastructure Time-Stamp Protocol (TSP) described in RFC 3161.

  • Time stamp policy OID: If the Time Stamping Server has time stamping policies configured, enter the OID of the policy to use into the Timestamping policy field. SSB will include this ID in the time stamping requests sent to the TSA.

NOTE: The time stamp requests are handled by a separate process in syslog-ng, message processing is not affected if the time stamping server is slow or cannot be accessed.

Related Documents

The document was helpful.

Select Rating

I easily found the information I needed.

Select Rating