The syslog-ng Store Box(SSB) appliance can separate a message consisting of key-value pairs (for example, Postfix log messages) into name-value pairs. The parsed values are automatically added to the metadata about the message, and you can display them on the SSB search interface as dynamic columns. You can specify the separator character to parse different log messages, for example, colon (:) to parse MySQL log messages, or the equal sign (=) for firewall logs. For details on when the key-value parser is executed related to other message processing operations, see the following list.

Caution: If the names of keys in the message is the same as the names of SSB soft macros, the value from the parsed message will overwrite the value of the macro. For example, the PROGRAM=value1, MESSAGE=value2 content will overwrite the ${PROGRAM} and ${MESSAGE} macros. To avoid overwriting such macros, use the prefix() option. Hard macros cannot be modified, so they will not be overwritten. For details on the macro types, see . The parser discards message sections that are not key=value pairs, even if they appear between key=value pairs that can be parsed.

1. Parse the message as a syslog message (unless message parsing is explicitly disabled for the source).

2. Classify the message using a pattern database.

3. Modify the message using rewrite rules (before filtering).

4. Filter the messages, for example, based on sender hostname or message content. If the message does not match the configured filter, syslog-ng Store Box(SSB) will not send it to the destination.

5. Parse the text of the message (that is, the ${MESSAGE} part) using a key-value parser or the sudo parser.

6. Modify the message using rewrite rules (after filtering and other parsing).

7. SSB sends the message to the destinations set in the log path. The destinations are local, optionally encrypted files on SSB, or remote servers, such as a database server.

To configure parsing key-value pairs

1. Navigate to Log > Parsers and select . A new parser is added to the list of parsers.

   Figure 199: Log > Parsers — Creating a key=value parser

   

2. Enter a name for the parser.

3. Enter the character that separates the keys from the values in the incoming messages into the Value separator character field. For example, if your messages look like key1:value1, key2:value2, key3:value3, enter :.

4. Enter the character or string that separates the key-value pairs from each other into the Pair separator string field. For example, if your messages look like key1:value1, key2:value2, key3:value3, enter ,. If you use a string, it can be a maximum of 20 characters long.

5. Enter a prefix before the key part of the parsed key-value pairs to help further processing into the Namespace field. For example, to insert the my-parsed-data prefix, enter my-parsed-data. Note the following points:

   • SSB automatically adds the .SDATA. prefix before the value you enter into the Namespace field. That way these values are automatically included in the structured data (SDATA) part of the log message if you forward the message using the IETF-syslog protocol.

   • SSB automatically adds a dot (.) character as a separator between the namespace and the key parsed from the message.

   For example, if you entered my-parsed-data as the namespace, and the keys in the message are key1, key2, and so on, then the full name of the macro that contains the parsed values is ${.SDATA.my-parsed-data.key1}, ${.SDATA.my-parsed-data.key2}, and so on. The parsed values are also automatically available as dynamic columns in the SSB search interface (the name of the column is the name of the macro).

6. Click .

7. Navigate to Log > Paths.

8. Select the path where you want to use the parser.

9. In the Parser field, Custom group, select the parser you want to use in this log path.

   Figure 200: Log > Paths — Using a key=value parser in the log path

   

10. Click .