Chat now with support
Chat with Support

syslog-ng Store Box 6.0.5 - Administration Guide

Preface Introduction The concepts of SSB The Welcome Wizard and the first login Basic settings User management and access control Managing SSB Configuring message sources Storing messages on SSB Forwarding messages from SSB Log paths: routing and processing messages Configuring syslog-ng options Searching log messages Searching the internal messages of SSB Classifying messages with pattern databases The SSB RPC API Troubleshooting SSB Security checklist for configuring SSB Glossary

Default logpaths in SSB

Two logpaths are available by default in SSB (see Log > Paths):

Figure 115: Log > Paths — Default logpaths of SSB

  • The first logpath collects the local messages of SSB. It sends every message of the web interface, the built-in syslog-ng server, and other internal components to the local logspace.

  • The second logpath collects messages sent to SSB using the default syslog sources (for details, see Default message sources in SSB) or via SNMP (for details, see Receiving SNMP messages). These messages are stored in the center logspace.

 NOTE:

Note that both default logpaths are marked as Final: if you create a new logpath that collects logs from the default sources, make sure to adjust the order of the logpaths, or disable the Final option for the default logpath.

Creating new logpaths

The following section describes how to create a new logpath.

To create a new logpath

  1. Navigate to Log > Paths and select . A new logpath is added to the list of logpaths.

  2. Select a source for the logpath from the Source field. Messages arriving to this source will be processed by this logpath. To add more sources to the logpath, select in the source field and repeat this step.

    Figure 116: Log > Paths — Creating a new logpath

    Remote sources receive messages from the network, while built-in sources are messages that originate on SSB. However, note that the SNMP source (for details, see Receiving SNMP messages) is listed in the built-in section.

    		 TIP:

    To process every message of every source, leave the source option on all. This is equivalent to using the catchall flag of syslog-ng.

  3. Select a destination for the logpath from the Destination field. Messages arriving to this source will be forwarded to this destination. To add more destinations to the logpath, select in the destination field and repeat this step.

    		 NOTE:

    Remote destinations forward the messages to external servers or databases and are configured on the Log > Destinations page (for details, see Forwarding messages from SSB).

    Local destinations store the messages locally on SSB and are configured on the Log > Logspaces page (for details, see Storing messages on SSB).

    If you do not want to store the messages arriving to this logpath, leave the Destination field on none.

    		 Caution:

    The none destination discards messages — messages sent only to this destination will be lost irrevocably.

  4. If you do not want other logpaths to process the messages sent to a destination by this logpath, select the Final option.

    The order of the logpaths is important, especially if you use the Final option in one or more destinations, because SSB evaluates logpaths in descending order. Use the , buttons to position the logpath if needed.

  5. To enable flow-control for this logpath, select the flow-control option. For details on how flow-control works, see Managing incoming and outgoing messages with flow-control.

    		 NOTE:

    As a result of toggling the flow-control status of the logpath, the output buffer size of the logpath's destination(s) will change. For the changes to take effect, navigate to Basic Settings > System > Service control and click Restart syslog-ng.

  6. If you do not want to send every message from the sources to the destinations, use filters. Select the filter to use from the Filter field, click , and configure the filter as needed. To apply more filters, click and select a new filter. Note that SSB sends only those messages to the destinations that pass every listed filter of the logpath. The available filters are described in Filtering messages.

    Figure 117: Log > Paths — Filtering log messages

  7. Click Commit. After that, the new logpath will start to collect log messages.

    		 TIP:

    If you do not want to activate the logpath immediately, deselect the Enable option.

Filtering messages

This section describes the filters that can be used in logpaths. Every filter can be used to select (for example, priority is) or exclude (for example, priority is not) messages. The following filters are available:

  • facility: Select messages sent by a specific facility (for example, kernel).

  • host: Select messages sent by a specific host. Enter the a hostname, IP address, or a POSIX (extended) regular expression.

  • message: Select messages containing a specific keyword or POSIX (extended) regular expression in the text of the log message (excluding the headers).

  • priority: Select messages of a specific priority.

  • program: Select messages sent by a specific application. Enter the name of the application or a POSIX (extended) regular expression.

  • sender: Filter on the address of the host that sent the message to SSB.

    		NOTE:

    To be able to use this filter, as a prerequisite, you must have a hostlist defined. For more information, see Creating hostlist policies.

    		 NOTE:

    When using the host, message, and program filters, remember to escape special characters. The characters ()[]{}.*?+^$|\ are treated as special symbols and have to be escaped with a backslash (\) in order to be interpreted as literal characters.

    		 NOTE:

    The effect of the sender and the host filters is the same if every client sends the logs directly to SSB. But if SSB receives messages from relays, then the host filter applies to the address of the clients, while the sender applies to the address of the relays.

If multiple filters are set for a logpath, only messages complying to every filter are sent to the destinations. (In other words, filters are added using the logical AND operation.)

Figure 118: Log > Paths — Using custom filters

If you need more complex filtering in your logpath, select the of the logpath and enter a custom filter into the appearing field. The contents of the Custom filter field are pasted into the filter() parameter of the syslog-ng logpath definition.

When defining custom filters, you can use regular expressions. By default, custom filters use POSIX-style (extended) regular expressions.

 NOTE:

When using POSIX regular expressions, the characters ()[]{}.*?+^$|\ are used as special symbols. Depending on how you want to use these characters and which quotation mark you use, these characters must be used differently, as summarized below:

  • When enclosing strings between double-quotes ("string"), the string is interpreted and you have to escape special characters, that is, prefix them with a backslash (\) if they are meant literally.

  • Strings between single quotes ('string') are treated as literals and are not interpreted at all, so you do not have to escape special characters.

To use other expression types, add the type() option after the regular expression. For example:

message("([0-9]+)=\\1" type("pcre"))

In this example, a PCRE regular expression with backreference is used and a match is returned if the message contains identical numbers separated by the equal sign (=). For example:

123=123

Replace message parts or create new macros with rewrite rules

SSB can rewrite parts of the messages using rewrite rules. Almost all part (macro) of the message can be rewritten. The rules use a key-value pair format.

The Replace with value completely replaces the old value of the message part. If the message part does not already exist, SSB automatically creates it. If you want to perform search and replace in the text of the log message, see Find and replace the text of the log message instead.

Note that you cannot change the values of hard macros in rewrite rules. For the list of hard macros, see Section Hard vs. soft macros in The syslog-ng Premium Edition 7.0.32 Administrator Guide.

To replace message parts or create new macros with rewrite rules

  1. Navigate to Log > Paths.

  2. Select the path(s) where you want to use rewrite rules.

  3. In the Rewrites section, click to add a new rewrite rule. Rewrite rules can be applied before filtering, or after filtering.

    The sequence of filtering and rewrite rules depends on how it was specified in the logpath. The sequence of the process is the following:

    1. Parse the message as a syslog message (unless message parsing is explicitly disabled for the source).

    2. Classify the message using a pattern database.

    3. Modify the message using rewrite rules (before filtering).

    4. Filter the messages, for example, based on sender hostname or message content. If the message does not match the configured filter, SSB will not send it to the destination.

    5. Parse the text of the message (that is, the ${MESSAGE} part) using a key-value parser or the sudo parser.

    6. Modify the message using rewrite rules (after filtering and other parsing).

    7. SSB sends the message to the destinations set in the logpath. The destinations are local, optionally encrypted files on SSB, or remote servers, such as a database server.

    Figure 119: Log > Paths — Modifying messages using rewrite

  4. Enter the part of the message to rewrite into the In Message part field. For example, MESSAGE, HOST, .SDATA.meta.custom. If the specified field does not exist, it is automatically created and set to the Replace with field.

  5. Enter the value of the message part after rewriting into the Replace with field. To use macros, begin with a $ sign and enclose the name of the macro between braces, for example ${MSG}, ${.SDATA.meta.custom}.

    		 NOTE:

  6. Click Commit.

Related Documents

The document was helpful.

Select Rating

I easily found the information I needed.

Select Rating