Example of conditional rewrites; combining a filter with a rewrite. (4372663)

Example of conditional rewrites; combining a filter with a rewrite.

How might one utilize a filter and a conditional rewrite to handle messages of different sizes in an alternate way?

*** NOTE the below is abbreviated example code portions only, and not a full configuration.

The following portions of a configuration file show how a filter can be combined with a conditional rewrite. In this example, lines <= 90 will be adjusted and have the word 'large' changed to 'audit'.
 

source s_largetest { network(ip(0.0.0.0) port (8888) transport("tcp")  );
        };

filter f_largefilter {
        "$(length ${MESSAGE})" >= "90"
        };

rewrite r_rewrite {
        subst('large', 'audit', value("MESSAGE") condition(filter(f_largefilter)));
        };

destination d_largelocalfile {file("/var/log/test_messages.txt");
        };

log {
        source(s_largetest);
        rewrite(r_rewrite);
        destination(d_largelocalfile);
};

Reviewing how the above would work with the following two messages:
 

<34>1 2023-08-08T12:34:56.123Z mymachine myapp 1234 mymsgid - This is an example log message, the word at the end will be changed: large
<34>1 2023-08-08T13:14:15:167Z short myapp 1234 mymsgid - Short - 88

The conditional rewrite in the above example will only act if the message is greater than or equal to 90 characters; if so then the word 'large' should be changed to 'audit' and the output would be:
 

cat /var/log/test_messages.txt
Aug  9 14:15:33 localhost 1 2023-08-08T12:34:56.123Z mymachine myapp 1234 mymsgid - This is an example log message, the word at the end will be changed: audit
Aug  9 14:15:33 localhost 1 2023-08-08T13:14:15:167Z short myapp 1234 mymsgid - Short – 88

In addition to the above, it would be possible to send the results  to different destinations (shorter messages to d_shortlocalfile and long messages to d_largelocalfile) if desired with a modification such as this (after also creating a d_shortlocalfile):

log {
        source(s_largetest);
        filter(f_largefilter);
        destination(d_largelocalfile);
        flags(final);
};

log {
        source(s_largetest);
        destination(d_shortlocalfile);

In the above example, a long message would be evaluated by the filter, and if it matches the criteria it will be written into d_largelocalfile. The flags(final) will ensure it is not processed further.
A shorter message would not match the filter, so would be handled by the next log statement.


 

The Administration Guide has further detail regarding conditional rewrites: https://support.oneidentity.com/technical-documents/syslog-ng-premium-edition/7.0.32/administration-guide/84#TOPIC-1925652