Chat now with support
Chat with Support

syslog-ng Store Box 6.1.0 - Administration Guide

Preface Introduction The concepts of SSB The Welcome Wizard and the first login Basic settings User management and access control Managing SSB Configuring message sources Storing messages on SSB Forwarding messages from SSB Log paths: routing and processing messages Configuring syslog-ng options Searching log messages Searching the internal messages of SSB Classifying messages with pattern databases The SSB RPC API Monitoring SSB Troubleshooting SSB Security checklist for configuring SSB

Metadata collected about log messages

The following information is available about the log messages:

  • Processed Timestamp: The date when SSB received the log message in YEAR-MONTH-DAY HOUR:MINUTE:SECOND format.

  • Timestamp: The timestamp received in the message — the time when the log message was created in YEAR-MONTH-DAY HOUR:MINUTE:SECOND format.

  • Facility: The facility that sent the message.

  • Priority: The priority value of the message.

  • Program: The application that created the message.

  • Pid: The program identifier of the application that created the message.

  • Host: The IP address or hostname of the client that sent the message to SSB.

  • Message: The text of the log message.

  • Tag: Tags assigned to the message matching certain pattern database rules.

  • Id: Unique ID of the message.

  • classifier.rule_id: ID of the pattern database rule that matched the message.

  • classifier.class: Description of the pattern database rule that matched the message.

  • Dynamic columns, created from additional name-value pairs, might also be available.

Using complex search queries

You can use wildcards and boolean expressions, and search specific parts of the log messages collected on SSB.

NOTE:

When searching log messages, the capabilities of the search engine depend on the delimiters used to index the particular logspace. By default, the indexer uses the following delimiter characters to separate the message into words (tokens): & : ~ ? ! [ ] = , ; ( ) ' ". For details on how to configure the delimiters used for indexing, see "Creating logstores" in the Administration Guide.

NOTE:

It is not possible to search for the whitespace ( ) character in the MESSAGE part of the log message, since it is a hard-coded delimiter character.

The following sections provide examples for different search queries:

Searching for exact matches and using complex queries

By default, SSB searches for keywords as whole words in the MESSAGE part of the log message and returns only exact matches.

Example: Searching for exact matches
Search expression example
Matches

example

Example

EXAMPLE

Does not match

examples

example.com

query-by-example

exam

Combining search keywords

You can use boolean operators - AND, OR, and NOT - to combine search keywords. Note that the boolean operators are case sensitive, and must be in all caps. More complex search expressions can also be constructed with parentheses.

Example: Combining keywords in search
Search expression keyword1 AND keyword2
Matches (returns log messages that contain both keywords)
Search expression keyword1 OR keyword2
Matches (returns log messages that contain at least one of the keywords)
Search expression keyword1 AND NOT keyword2
Matches (returns log messages that contain only keyword1)

To search for expressions that can be interpreted as boolean operators (for example: AND), use the following format: message:AND.

Example: Using parentheses in search

Use parentheses to create more complex search expressions:

Search expression (keyword1 OR keyword2) AND keyword3
Matches (returns log messages that contain either keyword1 and keyword3, or keyword2 and keyword3)
Using wildcard searches

You can use the ? and * wildcards in your search expressions.

Example: Using wildcard ? in search

The ? (question mark) wildcard means exactly one arbitrary character. Note that it does not work when trying to find non-UTF-8 or multibyte characters. If you want to search for these characters, the expression ?? might work, or you can use the * wildcard instead.

Search expression example?
Matches

example1

examples

Does not match

example.com

example12

query-by-example

example?

Search expression ?example?
Matches

1example2

Does not match

example.com

example12

query-by-example

Search expression example??
Matches

example12

Does not match

example.com

example1

query-by-example

Example: Using wildcard * in search

The * wildcard means 0 or more arbitrary characters. It finds non-UTF-8 and multibyte characters as well. Wildcard characters also work in any message part, for example, program:postfix*.

Search expression example*
Matches

example

examples

example.com

Does not match

query-by-example

example*

Search expression *example
Matches

example

query-by-example

Does not match

example.com

example12

Search expression *example*
Matches

example

query-by-example

example.com

example12

Example: Using combined wildcards in search

Wildcard characters can be combined.

Search expression ex?mple*
Matches

example1

examples

example.com

exemple.com

example12

Does not match

exmples

query-by-example

Searching for special characters

To search for the question mark (?), asterisk (*), backslash (\) or whitespace ( ) characters, you must prefix these characters with a backslash (\). Any character after a backslash is handled as a character to be searched for.

NOTE:

Delimiter characters are an exception to the rule. It is not possible to search for delimiter characters, even when they are prefixed.

Example: Searching for special characters

To search for a special character, use a backslash (\).

Search expression example\?
Matches

example?

Does not match

examples

example1

To search for the backslash character, use two backslashes (\\).

Search expression C:\\Windows
Matches

C:\Windows

Search expression nvpair:path=C:\\Program\ Files
Matches

C:\Program Files

Searching in a specific part of the message

You can search in a specific part of the message using the <type>: prefix. The message: (or msg:) prefix means the message part and can be omitted. For example, use the program: prefix to search for the name of an application, or use the host: prefix to search for a host name, and so on.

Example: Searching specific parts of messages
Search expression program:syslog-ng
Matches All log messages from the syslog-ng application.
Searching the name-value pairs of the message

You can search the structured data part of log messages using the nvpair: prefix. Use the = delimiter to separate the name and the value of structured data parameters, and remove the quote marks from the values.

Example: Searching the structured data part of messages
Search expression nvpair:.sdata.win@18372.4.event_type=Alert
Matches

All log messages where there is a win@18372.4 element with the event_type="Alert" parameter. For example:

[win@18372.4 EVENT_TYPE="Alert"]

Example: Using wildcard * to search the structured data

You can use the asterisk (*) wildcard to broaden the search to all structured data elements.

Search expression nvpair:*event_type=Alert*
Matches All log messages where the "event_type" name has the "Alert" value.
Example: Searching for parameter names

To search for a specific name, add the "=" character after the name.

Search expression nvpair:*event_type=*
Matches All log messages where an "event_type" name exists.
Example: Searching for parameter values

To search for a specific value, add the "=" character before the value.

Search expression nvpair:*=Alert
Matches All log messages where a name has the "Alert" value.

NOTE:

SSB only indexes the first 59 characters of every name-value pair (parameter). This has two consequences:

  • If the parameter is longer than 59 characters, an exact search might deliver multiple, imprecise results.

    Consider the following example. If the parameter is:

    .sdata.security.uid=2011-12-08T12:32:25.024+01:00-hostname-12345

    SSB indexes it only as:

    .sdata.security.uid=2011-12-08T12:32:25.024+01:00-hostname-

    This corresponds to the first 59 characters. As a result, searching for:

    nvpair:.sdata.security.uid=2011-12-08T12:32:25.024+01:00-hostname-12345

    returns all log messages that contain:

    .sdata.security.uid=2011-12-08T12:32:25.024+01:00-hostname-
  • Using wildcards might lead to the omission of certain messages from the search results.

    Using the same example as above, searching for the value:

    nvpair:*=2011-12-08T12:32:25.024+01:00-hostname-12345

    does not return any results (as the 12345 part was not indexed). Instead, you have to search for:

    nvpair:*=2011-12-08T12:32:25.024+01:00-hostname-*

    This, as explained above, might find multiple results.

Search performance tips

To decrease the load on SSB when searching and receive your search results faster, note the following points.

  • Use as small a time range as possible
  • Prefer AND instead of OR
  • Avoid unneeded wildcard characters, such as * and ?
  • Use wildcard characters at the end of the tokens if possible

Browsing encrypted logspaces

By default, you cannot browse encrypted logstores from the SSB web interface, because the required decryption keys are not available on SSB. To make browsing and searching encrypted logstores possible, SSB provides the following options:

NOTE:

Do not use SSB's own keys and certificates for encrypting or decrypting.

One Identity recommends:

  • Using 2048-bit RSA keys (or stronger).

  • Using the SHA-256 hash algorithm (or stronger) when creating the public key fingerprint.

Using persistent decryption keys

You can upload decryption keys and bind them to your account. The decryption keys are stored on SSB, but they are only made available for this user account, and can also be protected (encrypted) with a passphrase.

To use persistent decryption keys

  1. Select User menu > Private keystore. A pop-up window is displayed.

  2. Select Permanent > , then select Certificate > . A pop-up window is displayed.

    Figure 139: User menu > Private keystore — Adding decryption keys to the private keystore

  3. Paste or upload the certificate used to encrypt the logstore.

  4. Select Key > . A pop-up window is displayed.

  5. Paste or upload the private key of the certificate used to encrypt the logstore.

  6. Repeat Steps 2-5 to upload additional keys if needed.

  7. Select Security passphrase > Change, and enter a passphrase to protect the private keys.

    Figure 140: User menu > Private keystore — Securing the private keystore with a passphrase

  8. Click Apply.

Related Documents