The following describes how to customize the data displayed on the log message search interface.
To customize the data displayed on the log message search interface
-
Click Customize columns.
The parameters used for the columns when displaying log messages are listed under Displayed columns. All other available parameters are listed under Available static columns and Available dynamic columns.
Dynamic columns are created from structured data parameters (name-value pairs) in log messages stored on syslog-ng Store Box(SSB). Structured data parameters are detected and added to the list of customizable columns automatically. (For more information on the structured data part of log messages, see "The STRUCTURED-DATA message part" in the Administration Guide.)
NOTE: To export the search results into a CSV file, click 
on the action bar. Note that the CSV file includes all the static columns and the displayed dynamic columns.
Figure 215: Search > Logspaces > Customize columns — Customizing columns of the log message search interface
-
To add a static column to the Displayed columns, click 
.
-
To add a dynamic column to the Displayed columns, choose a name-value pair from Available dynamic columns and click .
The selected name generates a new, separate dynamic column with a <name> heading (where <name> is the name of the key). The relevant values are displayed in the cells of the respective column.
-
To remove parameters from the Visible columns, click 
.
-
To display the full content of each column (including the log messages), enable Show full content of columns.
The following information is available about the log messages:
-
Processed Timestamp: The date when syslog-ng Store Box(SSB) received the log message in YEAR-MONTH-DAY HOUR:MINUTE:SECOND format.
-
Timestamp: The time stamp received in the message — the time when the log message was created in YEAR-MONTH-DAY HOUR:MINUTE:SECOND format.
-
Facility: The facility that sent the message.
-
Priority: The priority value of the message.
-
Program: The application that created the message.
-
Pid: The program identifier of the application that created the message.
-
Host: The IP address or hostname of the client that sent the message to SSB.
-
Message: The text of the log message.
-
Tag: Tags assigned to the message matching certain pattern database rules.
-
Id: Unique ID of the message.
-
classifier.rule_id: ID of the pattern database rule that matched the message.
-
classifier.class: Description of the pattern database rule that matched the message.
-
Dynamic columns, created from additional name-value pairs, might also be available.
You can use wildcards and boolean expressions, and search specific parts of the log messages collected on syslog-ng Store Box(SSB).
NOTE: When searching log messages, the capabilities of the search engine depend on the delimiters used to index the particular logspace. By default, the indexer uses the following delimiter characters to separate the message into words (tokens): & : ~ ? ! [ ] = , ; ( ) ' ". For details on how to configure the delimiters used for indexing, see "Creating logstores" in the Administration Guide.
NOTE: It is not possible to search for the whitespace ( ) character in the MESSAGE part of the log message, since it is a hard-coded delimiter character.
The following sections provide examples for different search queries:
Searching for exact matches and using complex queries
By default, SSB searches for keywords as whole words in the MESSAGE part of the log message and returns only exact matches.
Example: Searching for exact matches
| Search expression |
example |
| Matches |
example
Example
EXAMPLE |
| Does not match |
examples
example.com
query-by-example
exam |
Combining search keywords
You can use boolean operators - AND, OR, and NOT - to combine search keywords. Note that the boolean operators are case sensitive, and must be in all caps. More complex search expressions can also be constructed with parentheses.
Example: Combining keywords in search
| Search expression |
keyword1 AND keyword2 |
| Matches |
(returns log messages that contain both keywords) |
| Search expression |
keyword1 OR keyword2 |
| Matches |
(returns log messages that contain at least one of the keywords) |
| Search expression |
keyword1 AND NOT keyword2 |
| Matches |
(returns log messages that contain only keyword1) |
To search for expressions that can be interpreted as boolean operators (for example: AND), use the following format: message:AND.
Example: Using parentheses in search
Use parentheses to create more complex search expressions:
| Search expression |
(keyword1 OR keyword2) AND keyword3 |
| Matches |
(returns log messages that contain either keyword1 and keyword3, or keyword2 and keyword3) |
Using wildcard searches
You can use the ? and * wildcards in your search expressions.
Example: Using wildcard ? in search
The ? (question mark) wildcard means exactly one arbitrary character. Note that it does not work when trying to find non-UTF-8 or multibyte characters. If you want to search for these characters, the expression ?? might work, or you can use the * wildcard instead.
| Search expression |
example? |
| Matches |
example1
examples |
| Does not match |
example.com
example12
query-by-example
example? |
| Search expression |
?example? |
| Matches |
1example2 |
| Does not match |
example.com
example12
query-by-example |
| Search expression |
example?? |
| Matches |
example12 |
| Does not match |
example.com
example1
query-by-example |
Example: Using wildcard * in search
The * wildcard means 0 or more arbitrary characters. It finds non-UTF-8 and multibyte characters as well. Wildcard characters also work in any message part, for example, program:postfix*.
| Search expression |
example* |
| Matches |
example
examples
example.com |
| Does not match |
query-by-example
example* |
| Search expression |
*example |
| Matches |
example
query-by-example |
| Does not match |
example.com
example12 |
| Search expression |
*example* |
| Matches |
example
query-by-example
example.com
example12 |
Example: Using combined wildcards in search
Wildcard characters can be combined.
| Search expression |
ex?mple* |
| Matches |
example1
examples
example.com
exemple.com
example12 |
| Does not match |
exmples
query-by-example |
Searching for special characters
To search for the question mark (?), asterisk (*), backslash (\) or whitespace () characters, you must prefix these characters with a backslash (\). Any character after a backslash is handled as a character to be searched for.
NOTE: Delimiter characters are an exception to the rule. It is not possible to search for delimiter characters, even when they are prefixed.
Example: Searching for special characters
To search for a special character, use a backslash (\).
| Search expression |
example\? |
| Matches |
example? |
| Does not match |
examples
example1 |
To search for the backslash character, use two backslashes (\\).
| Search expression |
C:\\Windows |
| Matches |
C:\Windows |
| Search expression |
nvpair:path=C:\\Program\ Files |
| Matches |
C:\Program Files |
Searching in a specific part of the message
You can search in a specific part of the message using the <type>: prefix. The message: (or msg:) prefix means the message part and can be omitted. For example, use the program: prefix to search for the name of an application, or use the host: prefix to search for a host name, and so on.
Example: Searching specific parts of messages
| Search expression |
program:syslog-ng |
| Matches |
All log messages from the syslog-ng application. |
Searching the name-value pairs of the message
You can search the structured data part of log messages using the nvpair: prefix. Use the = delimiter to separate the name and the value of structured data parameters, and remove the quote marks from the values.
Example: Searching the structured data part of messages
| Search expression |
nvpair:.sdata.win@18372.4.event_type=Alert |
| Matches |
All log messages where there is a win@18372.4 element with the event_type="Alert" parameter. For example:
[win@18372.4 EVENT_TYPE="Alert"] |
Example: Using wildcard * to search the structured data
You can use the asterisk (*) wildcard to broaden the search to all structured data elements.
| Search expression |
nvpair:*event_type=Alert* |
| Matches |
All log messages where the "event_type" name has the "Alert" value. |
Example: Searching for parameter names
To search for a specific name, add the "=" character after the name.
| Search expression |
nvpair:*event_type=* |
| Matches |
All log messages where an "event_type" name exists. |
Example: Searching for parameter values
To search for a specific value, add the "=" character before the value.
| Search expression |
nvpair:*=Alert |
| Matches |
All log messages where a name has the "Alert" value. |
NOTE: SSB only indexes the first 59 characters of every name-value pair (parameter). This has two consequences:
-
If the parameter is longer than 59 characters, an exact search might deliver multiple, imprecise results.
Consider the following example. If the parameter is:
.sdata.security.uid=2011-12-08T12:32:25.024+01:00-hostname-12345
SSB indexes it only as:
.sdata.security.uid=2011-12-08T12:32:25.024+01:00-hostname-
This corresponds to the first 59 characters. As a result, searching for:
nvpair:.sdata.security.uid=2011-12-08T12:32:25.024+01:00-hostname-12345
returns all log messages that contain:
.sdata.security.uid=2011-12-08T12:32:25.024+01:00-hostname-
-
Using wildcards might lead to the omission of certain messages from the search results.
Using the same example as above, searching for the value:
nvpair:*=2011-12-08T12:32:25.024+01:00-hostname-12345
does not return any results (as the 12345 part was not indexed). Instead, you have to search for:
nvpair:*=2011-12-08T12:32:25.024+01:00-hostname-*
This, as explained above, might find multiple results.
Search performance tips
To decrease the load on SSB when searching and receive your search results faster, note the following points.
- Use as small a time range as possible
- Prefer AND instead of OR
- Avoid unneeded wildcard characters, such as * and ?
- Use wildcard characters at the end of the tokens if possible
By default, you cannot browse encrypted logstores from the syslog-ng Store Box(SSB) web interface, because the required decryption keys are not available on SSB. To make browsing and searching encrypted logstores possible, SSB provides the following options:
NOTE: Do not use SSB's own keys and certificates for encrypting or decrypting.
One Identity recommends:
