The destination is aimed at a fully controlled local, or near-local, trusted SMTP server. The goal is to send mail to trusted recipients, through a controlled channel. It hands mails over to an SMTP server, and that is all it does, therefore the resulting solution is as reliable as sending an email in general. For example, syslog-ng PE does not verify whether the recipient exists.
The smtp() driver sends email messages triggered by log messages. The smtp() driver uses SMTP, without needing external applications. You can customize the main fields of the email, add extra headers, send the email to multiple recipients, and so on.
The subject(), body(), and header() fields may include macros which get expanded in the email. For more information on available macros see Macros of syslog-ng PE.
The smtp() driver has the following required parameters: host(), port(), from(), to(), subject(), and body(). For the list of available optional parameters, see smtp() destination options.
NOTE: To use this destination, syslog-ng Premium Edition (syslog-ng PE) must run in server mode. Typically, only the central syslog-ng PE server uses this destination. For more information on the server mode, see Server mode.
NOTE: The smtp() destination driver is available only in 5 F2 and later.
Declaration
smtp(host() port() from() to() subject() body() options());
Example: Using the smtp() driver
The following example defines an smtp() destination using only the required parameters.
destination d_smtp {
smtp(
host("localhost")
port(25)
from("syslog-ng alert service" "noreply@example.com")
to("Admin #1" "admin1@example.com")
subject("[ALERT] Important log message of $LEVEL condition received from $HOST/$PROGRAM!")
body("Hi!\nThe syslog-ng alerting service detected the following important log message:\n $MSG\n-- \nsyslog-ng\n")
);
};
The following example sets some optional parameters as well.
destination d_smtp {
smtp(
host("localhost")
port(25)
from("syslog-ng alert service" "noreply@example.com")
to("Admin #1" "admin1@example.com")
to("Admin #2" "admin2@example.com")
cc("Admin BOSS" "admin.boss@example.com")
bcc("Blind CC" "blindcc@example.com")
subject("[ALERT] Important log message of $LEVEL condition received from $HOST/$PROGRAM!")
body("Hi!\nThe syslog-ng alerting service detected the following important log message:\n $MSG\n-- \nsyslog-ng\n")
header("X-Program", "$PROGRAM")
);
};
Example: Simple email alerting with the smtp() driver
The following example sends an email alert if the eth0 network interface of the host is down.
filter f_linkdown {
match("eth0: link down" value("MESSAGE"));
};
destination d_alert {
smtp(
host("localhost") port(25)
from("syslog-ng alert service" "syslog@localhost")
reply-to("Admins" "root@localhost")
to("Ennekem" "me@localhost")
subject("[SYSLOG ALERT]: eth0 link down")
body("Syslog received an alert:\n$MSG")
);
};
log {
source(s_local);
filter(f_linkdown);
destination(d_alert);
};
The smtp() sends email messages using SMTP, without needing external applications. The smtp() destination has the following options:
body()
| Type: |
string |
| Default: |
n/a |
Description: The BODY field of the email. You can also use macros in the string. Use \n to start a new line. For example:
body("syslog-ng PE received the following alert from $HOST:\n$MSG")
bcc()
| Type: |
string |
| Default: |
n/a |
Description: The BCC recipient of the email (contents of the BCC field). You can specify the email address, or the name and the email address. Set the bcc() option multiple times to send the email to multiple recipients. For example: bcc("admin@example.com") or bcc("Admin" "admin@example.com") or bcc("Admin" "admin@example.com") bcc("Admin2" "admin2@example.com")
You can also use macros to set the value of this parameter.
cc()
| Type: |
string |
| Default: |
n/a |
Description: The CC recipient of the email (contents of the CC field). You can specify the email address, or the name and the email address. Set the cc() option multiple times to send the email to multiple recipients. For example: cc("admin@example.com") or cc("Admin" "admin@example.com") or cc("Admin" "admin@example.com") cc("Admin2" "admin2@example.com")
You can also use macros to set the value of this parameter.
disk-buffer()
Description: This option enables putting outgoing messages into the disk-buffer file of the destination to avoid message loss in case of a system failure on the destination side. It has the following options:
| dir() |
| Type: |
string |
| Default: |
N/A |
|
Description: Defines the folder where the disk-buffer files are stored.
Note that changing the value the dir() option will not move or copy existing files from the old directory to the new one.
|
|
Caution:
When creating a new dir() option for a disk-buffer file, or modifying an existing one, make sure you delete the persist file.
syslog-ng PE creates disk-buffer files based on the path recorded in the persist file. Therefore, if the persist file is not deleted after modifying the dir() option, then following a restart, syslog-ng PE will look for or create disk-buffer files in their old location. To ensure that syslog-ng PE uses the new dir() setting, the persist file must not contain any information about the destinations which the disk-buffer file in question belongs to. | |
| disk-buf-size() |
| Type: |
number [bytes] |
| Default: |
|
| Description: This is a required option. The maximum size of the disk-buffer file in bytes. The minimum value is 1048576 bytes. If you set a smaller value, the minimum value will be used automatically. It replaces the old log-disk-fifo-size() option. |
| mem-buf-length() |
| Type: |
number [messages] |
| Default: |
10000 |
| Description: Use this option if the option reliable() is set to no. This option contains the number of messages stored in overflow queue. It replaces the old log-fifo-size() option. It inherits the value of the global log-fifo-size() option if provided. If it is not provided, the default value is 10000 messages. Note that this option will be ignored if the option reliable() is set to yes. |
| mem-buf-size() |
| Type: |
number [bytes] |
| Default: |
163840000 |
| Description: Use this option if the option reliable() is set to yes. This option contains the size of the messages in bytes that is used in the memory part of the disk-buffer file. It replaces the old log-fifo-size() option. It does not inherit the value of the global log-fifo-size() option, even if it is provided. Note that this option will be ignored if the option reliable() is set to no. |
| quot-size() |
| Type: |
number [messages] |
| Default: |
1000 |
|
Description: The number of messages stored in the output buffer of the destination. Note that if you change the value of this option and the disk-buffer file already exists, the change will take effect when the disk-buffer file becomes empty. |
| reliable() |
| Type: |
yes|no |
| Default: |
no |
|
Description: If set to yes, syslog-ng PE cannot lose logs in case of reload/restart, unreachable destination or syslog-ng PE crash. This solution provides a slower, but reliable disk-buffer option. It is created and initialized at startup and gradually grows as new messages arrive. If set to no, the normal disk-buffer option will be used. This provides a faster, but less reliable disk-buffer option.
|
|
Caution: Hazard of data loss!
If you change the value of reliable() option when there are messages in the disk-buffer file, the messages stored in the disk-buffer file will be lost. | |
| truncate-size-ratio() |
| Type: |
number (for percentage) between 0 and 1 |
| Default: |
0.1 (10%) |
|
Description: Limits the truncation of the disk-buffer file. Truncating the disk-buffer file can slow down disk I/O operations, but it saves disk space. As a result, syslog-ng PE only truncates the file if the possible disk gain is more than truncate-size-ratio() times disk-buf-size().
|
|
Caution:
One Identity recommends that you do not modify the value of the truncate-size-ratio() option unless you are fully aware of the potential performance implications. | |
Example: Examples for using disk-buffer()
In the following case, reliable disk-buffer() is used.
destination d_demo {
network("127.0.0.1"
port(3333)
disk-buffer(
mem-buf-size(10000)
disk-buf-size(2000000)
reliable(yes)
dir("/tmp/disk-buffer")
)
);
};
In the following case normal disk-buffer() is used.
destination d_demo {
network("127.0.0.1"
port(3333)
disk-buffer(
mem-buf-length(10000)
disk-buf-size(2000000)
reliable(no)
dir("/tmp/disk-buffer")
)
);
};
from()
| Type: |
string |
| Default: |
n/a |
Description: The sender of the email (contents of the FROM field). You can specify the email address, or the name and the email address. For example:
from("admin@example.com")
or
from("Admin" "admin@example.com")
If you specify the from() option multiple times, the last value will be used. Instead of the from() option, you can also use sender(), which is just an alias of the from() option.
You can also use macros to set the value of this parameter.
header()
| Type: |
string |
| Default: |
n/a |
Description: Adds an extra header to the email with the specified name and content. The first parameter sets the name of the header, the second one its value. The value of the header can contain macros. Set the header() option multiple times to add multiple headers. For example:
header("X-Program", "$PROGRAM")
When using the header option, note the following points:
-
Do not use the header() option to set the values of headers that have dedicated options. Use it only to add extra headers.
-
If you set the same custom header multiple times, only the first will be added to the email, other occurrences will be ignored.
-
It is not possible to set the DATE, Return-Path, Original-Recipient, Content-*, MIME-*, Resent-*, Received headers.
host()
| Type: |
hostname or IP address |
| Default: |
n/a |
Description: Hostname or IP address of the SMTP server.
NOTE: If you specify host="localhost", syslog-ng PE will use a socket to connect to the local SMTP server. Use host="127.0.0.1" to force TCP communication between syslog-ng PE and the local SMTP server.
log-fifo-size()
| Type: |
number |
| Default: |
Use global setting. |
Description: The number of messages that the output queue can store.
port()
Description: The port number of the SMTP server.
reply-to()
| Type: |
string |
| Default: |
n/a |
Description: Replies of the recipient will be sent to this address (contents of the REPLY-TO field). You can specify the email address, or the name and the email address. Set the reply-to() option multiple times to send the email to multiple recipients. For example: reply-to("admin@example.com") or reply-to("Admin" "admin@example.com") or reply-to("Admin" "admin@example.com") reply-to("Admin2" "admin2@example.com")
You can also use macros to set the value of this parameter.
retries()
| Type: |
number [of attempts] |
| Default: |
3 |
Description: The number of times syslog-ng PE attempts to send a message to this destination. If syslog-ng PE could not send a message, it will try again until the number of attempts reaches retries(), then drops the message.
subject()
| Type: |
string |
| Default: |
n/a |
Description: The SUBJECT field of the email. You can also use macros. For example:
subject("[SYSLOG ALERT]: Critical error message received from $HOST")
If you specify the subject() option multiple times, the last value will be used.
throttle()
Description: Sets the maximum number of messages sent to the destination per second. Use this output-rate-limiting functionality only when using the disk-buffer option as well to avoid the risk of losing messages. Specifying 0 or a lower value sets the output limit to unlimited.
to()
| Type: |
string |
| Default: |
localhost |
Description: The recipient of the email (contents of the TO field). You can specify the email address, or the name and the email address. Set the to() option multiple times to send the email to multiple recipients. For example: to("admin@example.com") or to("Admin" "admin@example.com") or to("Admin" "admin@example.com") to("Admin2" "admin2@example.com")
You can also use macros to set the value of this parameter.
Version 7.0.12 of syslog-ng PE can directly post log messages to a Splunk deployment using the HTTP Event Collector (HEC) over the HTTP and Secure HTTP (HTTPS) protocols.
HTTPS connection, as well as password- and certificate-based authentication is supported. The content of the events is sent in JSON format.
NOTE: Typically, only the central syslog-ng PE server uses this destination. For more information on the server mode, see Server mode.
Declaration
d_splunk_hec {
splunk_hec(
#mandatory
index("<splunk-index-to-store-messages>")
token("<event-collector-tokens>")
url("http://<your-splunk-server>:8088/services/collector/event")
);
};
Prerequisites
-
On your Splunk deployment, you must enable HTTP Event Collector (HEC).
-
On your Splunk deployment, you must create a token for syslog-ng PE. You must use this token in the token() option of your splunk-hec() destination. We recommend to use the syslog source type for the token.
For details, see Set up and use HTTP Event Collector in Splunk Web.
Example: Sending log data to Splunk
The following example defines a splunk-hec() destination.
d_splunk_hec {
splunk_hec(
# mandatory
index("<splunk-index-to-store-messages>")
token("<event-collector-tokens>")
url("http://<your-splunk-server>:8088/services/collector/event")
# optional
batch_lines(25)
workers(4)
source("syslog-ng")
sourcetype("${.app.name:-syslog}")
delimiter("\n")
time("$S_UNIXTIME.$S_MSEC")
host("$HOST")
event("$S_ISODATE $HOST $MSGHDR$MSG\n")
timeout(10));
);
};
log {
source(s_file);
destination(d_splunk_hec);
flags(flow-control);
};
The splunk-hec() destination automatically sends multiple log messages in a single HTTP request, increasing the rate of messages that your Splunk deployment can consume. For details on adjusting and fine-tuning the batch mode of the splunk-hec() destination, see the following section.
Batch size
The batch-lines(), batch-lines(), and batch-timeout() options of the destination determine how many log messages syslog-ng PE sends in a batch. The batch-lines() option determines the maximum number of messages syslog-ng PE puts in a batch in. This can be limited based on size and time:
-
syslog-ng PE sends a batch every batch-timeout() milliseconds, even if the number of messages in the batch is less than batch-lines(). This ensures that the destination receives every message in a timely manner even if suddenly there are no more messages.
-
syslog-ng PE sends the batch if the total size of the messages in the batch reaches batch-bytes() bytes.
To increase the performance of the destination, increase the number of worker threads for the destination using the workers() option, or adjust the batch-bytes(), batch-lines(), batch-timeout() options.
Example: HTTP batch mode
In the following example, a batch consists of 100 messages, or a maximum of 512 kilobytes, and is sent every 20 seconds (20000 milliseconds).
destination d_splunk-hec {
splunk-hec(url("http://your-splunk-server:8088/services/collector/event")
index("<splunk-index-to-store-messages>")
token("<event-collector-tokens>")
url("http://your-splunk-server:8088/services/collector/event")
batch-lines(100)
batch-bytes(512Kb)
batch-timeout(10000)
);
};
Load balancing between multiple Splunk indexers
Starting with version 7.0.12, you can specify multiple URLs, for example, url("site1" "site2"). In this case, syslog-ng PE sends log messages to the specified URLs in a load-balance fashion. This means that syslog-ng PE sends each message to only one URL. For example, you can use this to send the messages to a set of ingestion nodes or indexers of your SIEM solution if a single node cannot handle the load. Note that the order of the messages as they arrive on the servers can differ from the order syslog-ng PE has received them, so use load-balancing only if your server can use the timestamp from the messages. If the server uses the timestamp when it receives the messages, the order of the messages will be incorrect.
|
|
Caution:
If you set multiple URLs in the url() option, set the persist-name() option as well to avoid data loss. |
Example: HTTP load balancing
The following destination sends log messages to 3 different Splunk indexer nodes. Each node is assigned a separate worker thread. A batch consists of 100 messages, or a maximum of 512 kilobytes, and is sent every 20 seconds (20000 milliseconds).
destination d_splunk-hec {
splunk-hec(url("http://your-splunk-server1:8088/services/collector/event" "http://your-splunk-server2:8088/services/collector/event" "http://your-splunk-server3:8088/services/collector/event")
batch-lines(100)
batch-bytes(512Kb)
batch-timeout(20000)
persist-name("d_splunk-hec-load-balance")
);
};
If you are using load-balancing (that is, you have configured multiple servers in the url() option), increase the number of worker threads at least to the number of servers. For example, if you have set three URLs (url("site1" "site2" "site3")), set the workers() option to 3 or more.
