syslog-ng Open Source Edition 3.30 - Administration Guide

syslog() source options

The syslog() driver has the following options.

ca-dir()

Description: The name of a directory that contains a set of trusted CA certificates in PEM format. The CA certificate files have to be named after the 32-bit hash of the subject's name. This naming can be created using the c_rehash utility in openssl. For an example, see Configuring TLS on the syslog-ng clients. The syslog-ng OSE application uses the CA certificates in this directory to validate the certificate of the peer.

This option can be used together with the optional ca-file() option.

ca-file()

Description: Optional. The name of a file that contains a set of trusted CA certificates in PEM format. The syslog-ng OSE application uses the CA certificates in this file to validate the certificate of the peer.

Example format in configuration:

ca-file("/etc/pki/tls/certs/ca-bundle.crt")

dynamic-window-size()

Description: The size of the dynamic control window used during flow-control. For details on flow-control, see Managing incoming and outgoing messages with flow-control.

encoding()

Description: Specifies the character set (encoding, for example, UTF-8) of messages using the legacy BSD-syslog protocol. To list the available character sets on a host, execute the iconv -l command. For details on how encoding affects the size of the message, see Message size and encoding.

hook-commands()

Description: This option makes it possible to execute external programs when the relevant driver is initialized or torn down. The hook-commands() can be used with all source and destination drivers with the exception of the usertty() and internal() drivers.

Using the hook-commands() when syslog-ng OSE starts or stops

To execute an external program when syslog-ng OSE starts or stops, use the following options:

Using the hook-commands() when syslog-ng OSE reloads

To execute an external program when the syslog-ng OSE configuration is initiated or torn down, for example, on startup/shutdown or during a syslog-ng OSE reload, use the following options:

source {
   network(transport(udp)
	hook-commands(
          startup("iptables -I LOGCHAIN 1 -p udp --dport 514 -j ACCEPT")
          shutdown("iptables -D LOGCHAIN 1")
        )
     );
};

host-override()

Description: Replaces the ${HOST} part of the message with the parameter string.

ip() or localip()

Description: The IP address to bind to. By default, syslog-ng OSE listens on every available interface. Note that this is not the address where messages are accepted from.

If you specify a multicast bind address and use the udp transport, syslog-ng OSE automatically joins the necessary multicast group. TCP does not support multicasting.

ip-protocol()

Description: Determines the internet protocol version of the given driver (network() or syslog()). The possible values are 4 and 6, corresponding to IPv4 and IPv6. The default value is ip-protocol(4).

Note that listening on a port using IPv6 automatically means that you are also listening on that port using IPv4. That is, if you want to have receive messages on an IP-address/port pair using both IPv4 and IPv6, create a source that uses the ip-protocol(6). You cannot have two sources with the same IP-address/port pair, but with different ip-protocol() settings (it causes an Address already in use error).

For example, the following source receives messages on TCP, using the network() driver, on every available interface of the host on both IPv4 and IPv6.

source s_network_tcp { network( transport("tcp") ip("::") ip-protocol(6) port(601) ); };

ip-tos()

Description: Specifies the Type-of-Service value of outgoing packets.

ip-ttl()

Description: Specifies the Time-To-Live value of outgoing packets.

keep-alive()

Description: Specifies whether connections to sources should be closed when syslog-ng is forced to reload its configuration (upon the receipt of a SIGHUP signal). Note that this applies to the server (source) side of the syslog-ng connections, client-side (destination) connections are always reopened after receiving a HUP signal unless the keep-alive option is enabled for the destination.

keep-hostname()

Description: Enable or disable hostname rewriting.

• If enabled (keep-hostname(yes)), syslog-ng OSE assumes that the incoming log message was sent by the host specified in the HOST field of the message.

• If disabled (keep-hostname(no)), syslog-ng OSE rewrites the HOST field of the message, either to the IP address (if the use-dns() parameter is set to no), or to the hostname (if the use-dns() parameter is set to yes and the IP address can be resolved to a hostname) of the host sending the message to syslog-ng OSE. For details on using name resolution in syslog-ng OSE, see Using name resolution in syslog-ng.

  s

This option can be specified globally, and per-source as well. The local setting of the source overrides the global option if available.

keep-timestamp()

Description: Specifies whether syslog-ng should accept the timestamp received from the sending application or client. If disabled, the time of reception will be used instead. This option can be specified globally, and per-source as well. The local setting of the source overrides the global option if available.

listen-backlog()

Description: Available only for stream based transports (unix-stream, tcp, tls). In TCP, connections are treated incomplete until the three-way handshake is completed between the server and the client. Incomplete connection requests wait on the TCP port for the listener to accept the request. The listen-backlog() option sets the maximum number of incomplete connection requests. For example:

source s_network {
    network(
        ip("192.168.1.1")
        transport("tcp")
        listen-backlog(2048)
        );
};

log-fetch-limit()

Description: The maximum number of messages fetched from a source during a single poll loop. The destination queues might fill up before flow-control could stop reading if log-fetch-limit() is too high.

log-msg-size()

Description: Maximum length of an incoming message in bytes. This length includes the entire message (the data structure and individual fields). The maximal value that can be set is 268435456 bytes (256 MiB).

For messages using the IETF-syslog message format (RFC5424), the maximal size of the value of an SDATA field is 64 KiB.

For details on how encoding affects the size of the message, see Message size and encoding.

Uses the value of the global option if not specified.

max-connections()

Description: Specifies the maximum number of simultaneous connections.

pad-size()

Description: Specifies input padding. Some operating systems (such as HP-UX) pad all messages to block boundary. This option can be used to specify the block size. The syslog-ng OSE application will pad reads from the associated device to the number of bytes set in pad-size(). Mostly used on HP-UX where /dev/log is a named pipe and every write is padded to 2048 bytes. If pad-size() was given and the incoming message does not fit into pad-size(), syslog-ng will not read anymore from this pipe and displays the following error message:

Padding was set, and couldn't read enough bytes

port() or localport()

Description: The port number to bind to.

program-override()

Description: Replaces the ${PROGRAM} part of the message with the parameter string. For example, to mark every message coming from the kernel, include the program-override("kernel") option in the source containing /proc/kmsg.

so-broadcast()

Description: This option controls the SO_BROADCAST socket option required to make syslog-ng send messages to a broadcast address. For details, see the socket(7) manual page.

so-keepalive()

Description: Enables keep-alive messages, keeping the socket open. This only effects TCP and UNIX-stream sockets. For details, see the socket(7) manual page.

so-rcvbuf()

Description: Specifies the size of the socket receive buffer in bytes. For details, see the socket(7) manual page.

so-reuseport()

Description: Enables SO_REUSEPORT on systems that support it. When enabled, the kernel allows multiple UDP sockets to be bound to the same port, and the kernel load-balances incoming UDP datagrams to the sockets. The sockets are distributed based on the hash of (srcip, dstip, srcport, dstport), so the same listener should be receiving packets from the same endpoint. For example:

source {
        udp(so-reuseport(1) port(2000) persist-name("udp1"));
        udp(so-reuseport(1) port(2000) persist-name("udp2"));
        udp(so-reuseport(1) port(2000) persist-name("udp3"));
        udp(so-reuseport(1) port(2000) persist-name("udp4"));
};

Enables keep-alive messages, keeping the socket open. This only effects TCP and UNIX-stream sockets. For details, see the socket(7) manual page.

so-sndbuf()

Description: Specifies the size of the socket send buffer in bytes. For details, see the socket(7) manual page.

tags()

Description: Label the messages received from the source with custom tags. Tags must be unique, and enclosed between double quotes. When adding multiple tags, separate them with comma, for example, tags("dmz", "router"). This option is available only in syslog-ng 3.1 and later.

time-zone()

Description: The default timezone for messages read from the source. Applies only if no timezone is specified within the message itself.

The timezone can be specified by using the name, for example, time-zone("Europe/Budapest")), or as the timezone offset in +/-HH:MM format, for example, +01:00). On Linux and UNIX platforms, the valid timezone names are listed under the /usr/share/zoneinfo directory.

trim-large-messages()

Description: Determines what syslog-ng OSE does with incoming log messages that are received using the IETF-syslog protocol using the syslog() driver, and are longer than the value of log-msg-size(). Other drivers ignore the trim-large-messages() option.

• If set to no, syslog-ng OSE drops the incoming log message.

• If set to yes, syslog-ng OSE trims the incoming log message to the size set in log-msg-size(), and adds the trimmed tag to the message. The rest of the message is dropped. You can use the tag to filter on such messages.

  filter f_trimmed {
      tags("trimmed");
  };

  If syslog-ng OSE trims a log message, it sends a debug-level log message to its internal() source.

  As a result of trimming, a parser could fail to parse the trimmed message. For example, a trimmed JSON or XML message will not be valid JSON or XML.

Available in syslog-ng OSE version 3.21 and later.

Uses the value of the global option if not specified.

tls()

Description: This option sets various options related to TLS encryption, for example, key/certificate files and trusted CA locations. TLS can be used only with tcp-based transport protocols. For details, see TLS options.

use-dns()

Description: Enable or disable DNS usage. The persist_only option attempts to resolve hostnames locally from file (for example, from /etc/hosts). The syslog-ng OSE application blocks on DNS queries, so enabling DNS may lead to a Denial of Service attack. To prevent DoS, protect your syslog-ng network endpoint with firewall rules, and make sure that all hosts which may get to syslog-ng are resolvable. This option can be specified globally, and per-source as well. The local setting of the source overrides the global option if available.

use-fqdn()

Description: Add Fully Qualified Domain Name instead of short hostname. This option can be specified globally, and per-source as well. The local setting of the source overrides the global option if available.

system: Collecting the system-specific log messages of a platform

Starting with version 3.2, syslog-ng OSE can automatically collect the system-specific log messages of the host on a number of platforms using the system() driver. If the system() driver is included in the syslog-ng OSE configuration file, syslog-ng OSE automatically adds the following sources to the syslog-ng OSE configuration.

The system() driver is also used in the default configuration file of syslog-ng OSE. For details on the default configuration file, see Example: The default configuration file of syslog-ng OSE. Starting with syslog-ng OSE version 3.6, you can use the system-expand command-line utility (which is a shell script, located in the modules/system-source/ directory) to display the configuration that the system() source will use.

Starting with version 3.6, syslog-ng OSE parses messages complying with the Splunk Common Information Model (CIM) and marked with @cim as JSON messages (for example, the ulogd from the netfilter project can emit such messages). That way, you can forward such messages without losing any information to CIM-aware applications (for example, Splunk).

unix-dgram("/dev/log");

unix-dgram("/var/run/log");

unix-dgram("/var/run/logpriv" perm(0600));

file("/dev/klog" follow-freq(0) program-override("kernel") flags(no-parse));

unix-dgram("/var/run/log");

file("/dev/klog" follow-freq(0) program-override("kernel"));

pipe("/dev/log" pad-size(2048));

unix-dgram("/dev/log");

file("/proc/kmsg" program-override("kernel") flags(kernel));

sun-streams("/dev/log");

sun-streams("/dev/log" door("/etc/.syslog_door"));

sun-streams("/dev/log" door("/var/run/syslog_door"));

system() source options

The system() driver has the following options:

hook-commands()

Description: This option makes it possible to execute external programs when the relevant driver is initialized or torn down. The hook-commands() can be used with all source and destination drivers with the exception of the usertty() and internal() drivers.

Using the hook-commands() when syslog-ng OSE starts or stops

To execute an external program when syslog-ng OSE starts or stops, use the following options:

Using the hook-commands() when syslog-ng OSE reloads

To execute an external program when the syslog-ng OSE configuration is initiated or torn down, for example, on startup/shutdown or during a syslog-ng OSE reload, use the following options:

source {
   network(transport(udp)
	hook-commands(
          startup("iptables -I LOGCHAIN 1 -p udp --dport 514 -j ACCEPT")
          shutdown("iptables -D LOGCHAIN 1")
        )
     );
};

systemd-journal: Collecting messages from the systemd-journal system log storage

The systemd-journal() source is used on various Linux distributions, such as RHEL (from RHEL7) and CentOS. The systemd-journal() source driver can read the structured name-value format of the journald system service, making it easier to reach the custom fields in the message. By default, syslog-ng OSE adds the .journald. prefix to the name of every parsed value. For a list and description of name-value pairs that journald provides, see the documentation of journald for your platform (for example, man systemd.journal-fields).

The systemd-journal() source driver is designed to read only local messages through the systemd-journal API. It is not possible to set the location of the journal files, or the directories.

systemd-journal(options);

@version: 3.30

source s_journald {
    systemd-journal(prefix(".SDATA.journald."));
};

destination d_network {
    syslog("server.host");
};

log {
    source(s_journald);
    destination(d_network);
};

@version: 3.30

source s_journald {
    systemd-journal(prefix(".SDATA.journald."));
};

filter f_uid {"${.SDATA.journald._UID}" eq "1000"};

destination d_network {
    syslog("server.host");
};

log {
    source(s_journald);
    filter(f_uid);
    destination(d_network);
};

@version: 3.30

source s_local {
    systemd-journal(prefix("journald."));
};

destination d_network {
    network("server.host" template("$(format_json --scope rfc5424 --key journald.*)\n"));
};

log {
    source(s_local);
    destination(d_network);
};

The journal contains credential information about the process that sent the log message. The syslog-ng OSE application makes this information available in the following macros: