The syslog-ng Store Box(SSB) appliance can separate a message consisting of key-value pairs (for example, Postfix log messages) into name-value pairs. The parsed values are automatically added to the metadata about the message, and you can display them on the SSB search interface as dynamic columns. You can specify the separator character to parse different log messages, for example, colon (:) to parse MySQL log messages, or the equal sign (=) for firewall logs. For details on when the key-value parser is executed related to other message processing operations, see the following list.
-
Parse the message as a syslog message (unless message parsing is explicitly disabled for the source).
-
Classify the message using a pattern database.
-
Modify the message using rewrite rules (before filtering).
-
Filter the messages, for example, based on sender hostname or message content. If the message does not match the configured filter, syslog-ng Store Box(SSB) will not send it to the destination.
-
Parse the text of the message (that is, the ${MESSAGE} part) using a key-value parser or the sudo parser.
-
Modify the message using rewrite rules (after filtering and other parsing).
-
SSB sends the message to the destinations set in the log path. The destinations are local, optionally encrypted files on SSB, or remote servers, such as a database server.
NOTE: If a log message contains the same key multiple times (for example, key1=value1, key2=value2, key1=value3, key3=value4, key1=value5), then SSB stores only the last (rightmost) value for the key. Using the previous example, SSB will store the following pairs: key1=value5, key2=value2, key3=value4..
NOTE: The names of the keys can contain only the following characters: numbers (0-9), letters (a-z,A-Z), underscore (_), dot (.), hyphen (-). Other special characters are not permitted.
To configure parsing key-value pairs
-
Navigate to Log > Parsers and select
. A new parser is added to the list of parsers.
Figure 202: Log > Parsers — Creating a key=value parser
-
Enter a name for the parser.
-
Enter the character that separates the keys from the values in the incoming messages into the Value separator character field. For example, if your messages look like key1:value1, key2:value2, key3:value3, enter :.
- Enter the character or string that separates the key-value pairs from each other into the Pair separator string field. For example, if your messages look like key1:value1, key2:value2, key3:value3, enter ,. If you use a string, it can be a maximum of 20 characters long.
-
Enter a prefix before the key part of the parsed key-value pairs to help further processing into the Namespace field. For example, to insert the my-parsed-data prefix, enter my-parsed-data.
NOTE: Consider the following when configuring a prefix:
-
SSB automatically adds the .SDATA. prefix before the value you enter into the Namespace field. That way these values are automatically included in the structured data (SDATA) part of the log message if you forward the message using the IETF-syslog protocol.
-
SSB automatically adds a dot (.) character as a separator between the namespace and the key parsed from the message.
-
-
Click
.
-
Navigate to Log > Paths.
-
Select the path where you want to use the parser.
-
In the Parser field, Custom group, select the parser you want to use in this log path.
Figure 203: Log > Paths — Using a key=value parser in the log path
-
Click
