The Full template
The Full template stores messages in the ssb_sql_messages_${R_YEAR}_${R_MONTH} table. The following columns are created:
-
insert_time: The date when syslog-ng Store Box(SSB) received the message in Unixtime format.
-
rule_id: ID of the pattern database rule that matched the message.
-
__row_id: Identifier of the row.
-
date_time: The date the message was sent in YEAR-MONTH-DAY HOUR:MINUTE:SECOND format.
-
facility: The facility that sent the message.
-
priority: The priority level of the message.
-
sourceip: The IP address of the host that sent the message.
-
host: The IP address or hostname of the host were the message was generated.
-
program: The name of the application that generated the message.
-
pid: The ID number of the process that generated the message (this field is automatically set to zero if the PID is not included in the message).
-
message: The text of the log message.
The insert_time, rule_id, date_time, facility, host, sourceip, and program columns are indexed.
The Custom template
The Custom template allows you to specify the columns to use. Enter a name for the column, select its type, and specify its content using macros. For details on using macros, see Macros of syslog-ng PE in the syslog-ng PE Administration Guide and Hard versus soft macros in the syslog-ng PE Administration Guide.
Select the Indexed option if you want the database to index the column.
Forwarding log messages to remote servers
This section describes how to forward messages from syslog-ng Store Box(SSB) to a remote server.
To forward messages from SSB to a remote server
-
Navigate to Log > Destinations and select 
to create a new remote destination.
-
Select Remote host.
Figure 177: Log > Destinations — Creating server destinations
-
Enter the IP address or hostname of the remote server into the Address field. Enter the port where the server is accepting syslog messages into the Port field.
NOTE: The Address and Port pair must be unique for each remote destination.
-
Select the network protocol used to transfer the log messages from the Transport field. The UDP, TCP, and the encrypted TLS protocols are available. The UDP and TLS protocols have additional parameters.
When forwarding messages using UDP, the remote host will see the messages as if they originated from SSB. Select the Spoof source address option to make them seem to originate from their original sender.
|
|
Caution:
When using the Spoof source address option, SSB automatically truncates long messages to 1024 bytes, regardless of the Log > Options > Message size setting. |
For TLS, select a method to verify the identity of the remote host. The following options are available:
-
None: Do not request a certificate from the remote host, and accept any certificate if the host sends one.
-
Optional trusted: If the remote host sends a certificate, SSB checks if it is valid (not expired) and that the Common Name of the certificate contains the domain name or the IP address of the host. If these checks fail, SSB rejects the connection. However, SSB accepts the connection if the host does not send a certificate.
-
Optional untrusted: Accept any certificate shown by the remote host. Note that the host must show a certificate.
-
Required trusted (default setting): Verify the certificate of the remote host. Only valid certificates signed by a trusted certificate authority are accepted. See Uploading external certificates to SSB for details on importing CA certificates. Note that the Common Name of the certificate must contain the domain name or the IP address of the host.
-
Required untrusted: SSB requests a certificate from the remote host, and rejects the connection if no certificate is received. However, SSB accepts the connection if:
-
the certificate is not valid (expired), or
-
the Common Name of the certificate does not contain the domain name or the IP address of the host.
NOTE: See the documentation of the remote server application to determine which protocols are supported.
UDP is a highly unreliable protocol and a high amount of messages may be lost without notice during the transfer. Use TCP or TLS instead whenever possible.
-
Select the syslog protocol to use from the Syslog protocol field.
-
To use the legacy BSD-syslog protocol described in RFC 3164, select Legacy and specify the message template to use. Select Legacy to use the message format described in the RFC, or ISO date to replace the original time stamp with an ISO8061 compliant time stamp that includes year and timezone information. To customize the format of the message contents using macros, select Custom message part only, or Custom on-wire message to completely reformat the message (including the headers). For details on using macros, see Hard versus soft macros in the syslog-ng PE Administration Guide. If you have no special requirements, use the ISO date template.
-
Use the new IETF-syslog protocol.
NOTE: Consider the following when using the IETF-syslog protocol:
-
Most syslog applications and devices currently support only the legacy protocol. Consult the documentation of the remote server application to determine which protocols are supported.
-
If you need, you can customize the contents of the message using macros. However, for the IETF-syslog protocol, the header cannot be customized. For details on using macros, see Templates and macros in the syslog-ng Premium Edition Administration Guide.
-
If SSB would send several messages with identical content to the destination, it can send only a single message and a line Last message repeated n times.. Enter the number of seconds to wait for identical messages into the Suppress timeout field. This option corresponds to the suppress() parameter of syslog-ng.
-
To limit the maximum number of messages sent to the destination per second, enter the maximum number of messages into the Message throttle field. Use this output-rate-limiting functionality only when using disk-buffer as well to avoid the risk of losing messages. Specifying 0 or a lower value sets the output limit to unlimited. This option corresponds to the throttle() parameter of syslog-ng.
-
The time stamps of most log messages is accurate only to the second. The syslog-ng Store Box(SSB) appliance can include more accurate time stamps: set how many digits should be included in the Timestamp fractions of a second field. This option corresponds to the frac_digits() parameter of syslog-ng.
-
If the server and SSB are located in a different timezone and you use the Legacy message template (which does not include timezone information), select the timezone of the server from the Timezone field.
-
Set the size of the disk buffer (in Megabytes) in the Output disk buffer field. If the remote server becomes unavailable, SSB will buffer messages to the hard disk, and continue sending the messages when the remote server becomes available. This option corresponds to the log_disk_fifo_size() parameter of syslog-ng.
Note that SSB does not pre-allocate the hard disk required for the disk buffer, so make sure that the required disk space is available on SSB. For details on creating archiving policies and adjusting the disk-fillup prevention, see Archiving and cleanup and Preventing disk space fill up.
Example: Calculating disk buffer size
The size of the disk buffer you need depends on the rate of the incoming messages, the size of the messages, and the length of the network outage that you want to cover. For example:
-
SSB is receiving 15000 messages per second
-
On the average, one message is 250 bytes long
-
You estimate that the longest time the destination will be unavailable is 4 hours
In this case, you need a disk buffer for 250 [bytes] * 15000 [messages per second] * 4*60*60 [seconds] = 54000000000 [bytes], which is 54000 Megabytes (in other words, a bit over 50 GB).
-
Click 
.
-
To start sending messages to the destination, include the new destination in a logpath. For details, see Log paths: routing and processing messages.
Forwarding log messages to the Microsoft Azure Sentinel cloud
This section describes how to forward messages from syslog-ng Store Box (SSB) to the Microsoft Azure Sentinel cloud (Azure Sentinel).
Azure Sentinel is Microsoft's native cloud-based SIEM solution. Beside Microsoft's own cloud services, it can accept log messages from external sources. Microsoft provides 26 predefined Data connectors and a HTTP Data Collector API for further integrations.
From version 6.7.0, SSB uses the syslog-ng Premium Edition (syslog-ng PE) application's support to post messages to Azure Sentinel over HTTP and Secure HTTP (HTTPS).
For further details about how the application forwards messages to Azure Sentinel, see sentinel: Sending logs to the Microsoft Azure Sentinel cloud in the syslog-ng PE Administration Guide.
For more information about Azure Sentinel, see Microsoft Azure: Azure Sentinel documentation in the Microsoft Azure Documentation.
For more information about Data connectors used in Azure Sentinel, see Microsoft Azure: Connect data sources in the Microsoft Azure Documentation.
NOTE: This section and the other Azure Sentinel-related sections in this documentation are based on Azure Sentinel messaging service concepts and terminology. If you do not use the Azure Sentinel messaging service on a regular basis, One Identity recommends that you read the Azure Sentinel quick-start documentation to familiarize yourself with the messaging service's concepts and terminology before you continue reading these sections.
