syslog-ng Premium Edition 6.0.21 - Administration Guide

Description: Use POSIX regular expressions. If the type() parameter is not specified, syslog-ng uses POSIX regular expressions by default.

Posix regular expressions have the following flag options:

global: Usable only in rewrite rules: match for every occurrence of the expression, not only the first one.

ignore-case: Disable case-sensitivity.

store-matches: Store the matches of the regular expression into the $0, ... $255 variables. The $0 stores the entire match, $1 is the first group of the match (parentheses), and so on. Matches from the last filter expression can be referenced in regular expressions.

utf8: Use UTF-8 matching.

filter f_message { message("keyword" flags("utf8" "ignore-case") ); };

Description: Use Perl Compatible Regular Expressions (PCRE). Starting with syslog-ng PE version 3.1, PCRE expressions are supported on every platform.

PCRE regular expressions have the following flag options:

global: Usable only in rewrite rules: match for every occurrence of the expression, not only the first one.

ignore-case: Disable case-sensitivity.

store-matches: Store the matches of the regular expression into the $0, ... $255 variables. The $0 stores the entire match, $1 is the first group of the match (parentheses), and so on. Named matches (also called named subpatterns), for example (?<name>...), are stored as well. Matches from the last filter expression can be referenced in regular expressions.

unicode: Use Unicode support for UTF-8 matches: UTF-8 character sequences are handled as single characters.

utf8: An alias for the unicode flag.

rewrite r_rewrite_subst
        {subst("a*", "?", value("MESSAGE") type("pcre") flags("utf8" "global"));  };

Description: Match the strings literally, without regular expression support. By default, only identical strings are matched. For partial matches, use the flags("prefix") or the flags("substring") flags.

Description: Match the strings against a pattern containing '*' and '?' wildcards, without regular expression and character range support. The advantage of glob patterns to regular expressions is that globs can be processed much faster.

Description: Insert a prefix before the name part of the parsed name-value pairs to help further processing. For example:

Names starting with a dot (for example, .example) are reserved for use by syslog-ng PE. If you use such a macro name as the name of a parsed value, it will attempt to replace the original value of the macro (note that only soft macros can be overwritten, see the section called “Hard vs. soft macros” for details). To avoid such problems, use a prefix when naming the parsed values, for example, prefix(my-parsed-data.)

For example, to insert the postfix prefix when parsing Postfix log messages, use the prefix(.postfix.) option.

Description: The macro that contains the part of the message that the parser will process. It can also be a macro created by a previous parser of the log path. By default, the parser processes the entire message (${MESSAGE}).

Description: Specifies the character that separates the keys from the values. Default value: =

For example, to parse key:value pairs, use kv-parser(value-separator(":"));

Description: Extract only the specified subtree from the JSON message. Use the dot-notation to specify the subtree. The rest of the message will be ignored. For example, assuming that the incoming object is named msg, the json-parser(extract-prefix("foo.bar[5]")); syslog-ng PE parser is equivalent to the msg.foo.bar[5] javascript code. Note that the resulting expression must be a JSON object, so that syslog-ng PE can extract its members into name-value pairs.

This feature also works when the top-level object is an array, because you can use an array index at the first indirection level, for example: json-parser(extract-prefix("[5]")), which is equivalent to msg[5].

Description: Use a marker in case of mixed log messages, to identify JSON encoded messages for the parser.

Some logging implementations require a marker to be set before the JSON payload. The JSON parser is able to find these markers and parse the message only if it is present.

parser {
        json-parser(
            marker("@cee:")
            prefix(".cee.")
        );
    };

Description: Insert a prefix before the name part of the name-value pairs to help further processing. For example, if you forward the parsed messages using the IETF-syslog protocol, you can insert all the parsed data into the SDATA part of the message using the prefix(.SDATA.json.) .

Names starting with a dot (for example, .example) are reserved for use by syslog-ng PE. If you use such a macro name as the name of a parsed value, it will attempt to replace the original value of the macro (note that only soft macros can be overwritten, see the section called “Hard vs. soft macros” for details). To avoid such problems, use a prefix when naming the parsed values, for example, prefix(my-parsed-data.)

Description: The macro that contains the part of the message that the parser will process. It can also be a macro created by a previous parser of the log path. By default, this is empty and the parser processes the entire message (${MESSAGE}).