Starting with version 3.24 of the syslog-ng Open Source Edition (syslog-ng OSE) application, you can manipulate the timezone information of messages using rewrite rules. You can:
By default, these operations modify the date-related macros of the message that correspond to the date the message was sent (that is, the S_ macros). You can modify the dates when syslog-ng OSE has received the messages (that is, the R_ macros), but this is rarely needed. To do so, include the time-stamp(recvd) option in the operation, for example:
rewrite { fix-time-zone("EST5EDT" time-stamp(recvd)); };
Use the fix-time-zone() operation to correct the timezone of a message if it was parsed incorrectly for some reason, or if the client did not include any timezone information in the message. You can specify the new timezone as the name of a timezone, or as a template string. For example, use the following rewrite rule to set the timezone to EST5EDT:
rewrite { fix-time-zone("EST5EDT"); };
If you have lots of clients that do not send timezone information in the log messages, you can create a database file that stores the timezone of the clients, and feed this data to syslog-ng OSE using the add-contextual-data() feature. For details, see Adding metadata from an external file.
Use the guess-time-zone() operation attempts to set the timezone of the message automatically, using heuristics on the timestamps. Normally the syslog-ng OSE application performs this operation automatically when it parses the incoming message. Using this operation in a rewrite rule can be useful if you cannot parse the incoming message for some reason (and use the flags(no-parse) option in your source, but you want to set the timezone automatically later (for example, after you have preprocessed the message).
Using this operation is identical to using the flags(guess-timezone) flag in the source.
Use the set-time-zone() operation to set the timezone of the message to a specific value, that is to convert an existing timezone to a different one. This operation is identical to setting the time-zone() option in a destination or as a global option, but can be applied selectively to the messages using conditions.
Log messages of banking and e-commerce applications might include credit card numbers (Primary Account Number or PAN). According to privacy best practices and the requirements of the Payment Card Industry Data Security Standards (PCI-DSS), PAN must be rendered unreadable. The syslog-ng OSE application uses a regular expression to detect credit card numbers, and provides two ways to accomplish this: you can either mask the credit card numbers, or replace them with a hash. To mask the credit card numbers, use the credit-card-mask() or the credit-card-hash() rewrite rules in a log path.
Declaration:
@include "scl/rewrite/cc-mask.conf"
rewrite {
credit-card-mask(value("<message-field-to-process>"));
};
By default, these rewrite rules process the MESSAGE part of the log message.
credit-card-hash()
| Synopsis: |
credit-card-hash(value("<message-field-to-process>")) |
Description: Process the specified message field (by default, ${MESSAGE}), and replace any credit card numbers (Primary Account Number or PAN) with a 16-character-long hash. This hash is generated by calculating the SHA-1 hash of the credit card number, selecting the first 64 bits of this hash, and representing this 64 bits in 16 characters.
credit-card-mask()
| Synopsis: |
credit-card-mask(value("<message-field-to-process>")) |
Description: Process the specified message field (by default, ${MESSAGE}), and replace the 7-12th character of any credit card numbers (Primary Account Number or PAN) with asterisks (*). For example, syslog-ng OSE replaces the number 5542043004559005 with 554204******9005.
Filters and substitution rewrite rules can use regular expressions. In regular expressions, the characters ()[].*?+^$|\ are used as special symbols. Depending on how you want to use these characters and which quotation mark you use, these characters must be used differently, as summarized below.
-
Strings between single quotes ('string') are treated literally and are not interpreted at all, you do not have to escape special characters. For example, the output of '\x41' is \x41 (characters as follows: backslash, x(letter), 4(number), 1(number)). This makes writing and reading regular expressions much more simple: it is recommended to use single quotes when writing regular expressions.
-
When enclosing strings between double-quotes ("string"), the string is interpreted and you have to escape special characters, that is, to precede them with a backslash (\) character if they are meant literally. for example, the output of the "\x41" is simply the letter a. Therefore special characters like \(backslash) or "(quotation mark) must be escaped (\\ and \"). The following expressions are interpreted: \a, \n, \r, \t, \v. For example, the \$40 expression matches the $40 string. Backslashes have to be escaped as well if they are meant literally, for example, the \\d expression matches the \d string.
TIP: If you use single quotes, you do not need to escape the backslash, for example, match("\\.") is equivalent to match('\.').
-
Enclosing alphanumeric strings between double-quotes ("string") is not necessary, you can just omit the double-quotes. for example, when writing filters, match("sometext") and match(sometext) will both match for the sometext string.
NOTE: Only strings containing alphanumerical characters can be used without quotes or double quotes. If the string contains whitespace or any special characters (()[].*?+^$|\ or ;:#), you must use quotes or double quotes.
When using the ;:# characters, you must use quotes or double quotes, but escaping them is not required.
By default, all regular expressions are case sensitive. To disable the case sensitivity of the expression, add the flags(ignore-case) option to the regular expression.
filter demo_regexp_insensitive {
host("system" flags(ignore-case));
};
NOTE:Adding the flags(ignore-case) option to glob patterns does not disable case sensitivity.
The regular expressions can use up to 255 regexp matches (${1} ... ${255}), but only from the last filter and only if the flags("store-matches") flag was set for the filter. For case-insensitive searches, use the flags("ignore-case") option.
This chapter lists regular expressions supported by syslog-ng Open Source Edition (syslog-ng OSE) and their available supported type() and flags() options.
By default, syslog-ng OSE uses PCRE-style regular expressions. To use other expression types, add the type() option after the regular expression.
The syslog-ng OSE application supports the following regular expression type() options:
