Chat now with support
Chat with Support

syslog-ng Premium Edition 6.0.18 - Administration Guide

Preface Chapter 1. Introduction to syslog-ng Chapter 2. The concepts of syslog-ng Chapter 3. Installing syslog-ng Chapter 4. The syslog-ng PE quick-start guide Chapter 5. The syslog-ng PE configuration file Chapter 6. Collecting log messages — sources and source drivers Chapter 7. Sending and storing log messages — destinations and destination drivers Chapter 8. Routing messages: log paths, reliability, and filters Chapter 9. Global options of syslog-ng PE Chapter 10. TLS-encrypted message transfer Chapter 11. FIPS-compliant syslog-ng Chapter 12.  Reliable Log Transfer Protocol™ Chapter 13. Reliability and minimizing the loss of log messages Chapter 14. Manipulating messages Chapter 15. Parsing and segmenting structured messages Chapter 16. Processing message content with a pattern database Chapter 17. Statistics and metrics of syslog-ng Chapter 18. Multithreading and scaling in syslog-ng PE Chapter 19. Troubleshooting syslog-ng Chapter 20. Best practices and examples

Collecting the system-specific log messages of a platform

Starting with version 4 F1, syslog-ng PE can automatically collect the system-specific log messages of the host on a number of platforms using the system() driver. If the system() driver is included in the syslog-ng PE configuration file, syslog-ng PE automatically adds the following sources to the syslog-ng PE configuration.

NOTE:

syslog-ng PE versions 4.1-5.0 used an external script to generate the system() source, but this was problematic in certain situations, for example, when the host used a strict AppArmor profile. Therefore, the system() source is now generated internally in syslog-ng PE.

The system() driver is also used in the default configuration file of syslog-ng PE. For details on the default configuration file, see Example 4.1, “The default configuration file of syslog-ng PE”.

Caution:

If syslog-ng PE does not recognize the platform it is installed on, it does not add any sources.

Table 6.4. Sources automatically added by syslog-ng Premium Edition

Platform Message source
AIX
unix-dgram("/dev/log");
FreeBSD
unix-dgram("/var/run/log");
unix-dgram("/var/run/logpriv" perm(0600));
file("/dev/klog" follow-freq(0) program-override("kernel") flags(no-parse));
HP-UX
pipe("/dev/log" pad-size(2048));
Linux
unix-dgram("/dev/log");
file("/proc/kmsg" program-override("kernel") flags(kernel));
Solaris 8
sun-streams("/dev/log");
Solaris 9
sun-streams("/dev/log" door("/etc/.syslog_door"));
Solaris 10
sun-streams("/dev/log" door("/var/run/syslog_door"));

Collecting messages from the systemd-journal system log storage

The systemd-journal() source is used on various Linux distributions, such as RHEL (from RHEL7) and CentOS. The systemd-journal() source driver can read the structured name-value format of the journald system service, making it easier to reach the custom fields in the message.

The systemd-journal() source driver is designed to read only local messages through the systemd-journal API. It is not possible to set the location of the journal files, or the directories.

NOTE:

The log-msg-size() option is not applicable for this source. Use the max-field-size() option instead.

NOTE:

This source will not handle the following cases:

  • corrupted journal file

  • incorrect journal configuration

  • any other journald-related bugs

NOTE:

If you are using RHEL-7, the default source in the configuration is systemd-journal() instead of unix-dgram("/dev/log") and file("/proc/kmsg"). If you are using unix-dgram("/dev/log") or unix-stream("/dev/log") in your configuration as a source, syslog-ng PE will revert to using systemd-journal() instead.

Caution:

Only one systemd-journal() source can be configured in the configuration file. If there are more than one systemd-journal() sources configured, syslog-ng PE will not start.

Declaration: 

systemd-journal(options);

Example 6.37. Sending all fields through syslog protocol using the systemd-journal() driver

To send all fields through the syslog protocol, enter the prefix in the following format: ".SDATA.<name>".

@version: 6.0

source s_journald {
	systemd-journal(prefix(".SDATA.journald."));
};

destination d_network {
	syslog("server.host");
};

log {
	source(s_journald);
	destination(d_network);
};

Example 6.38. Filtering for a specific field using the systemd-journal() driver

@version: 6.0

source s_journald {
	systemd-journal(prefix(".SDATA.journald."));
};

filter f_uid {"${.SDATA.journald._UID}" eq "1000"};

destination d_network {
	syslog("server.host");
};

log {
	source(s_journald);
	filter(f_uid);
	destination(d_network);
};

Example 6.39. Sending all fields in value-pairs using the systemd-journal() driver

@version: 6.0

source s_local {
	systemd-journal(prefix("journald."));
};

destination d_network {
	network("server.host" template("$(format_json --scope rfc5424 --key journald.*)\n"));
};

log {
	source(s_local);
	destination(d_network);
};

The journal contains credential information about the process that sent the log message. The syslog-ng PE application makes this information available in the following macros:

Journald field syslog-ng predefined macro
MESSAGE $MESSAGE
_HOSTNAME $HOST
_PID $PID
_COMM if does not exist SYSLOG_IDENTIFIER $PROGRAM
SYSLOG_FACILITY $FACILITY_NUM
PRIORITY $LEVEL_NUM

systemd-journal() source options

The systemd-journal() driver has the following options:

default-facility()
Type: facility string
Default: local0

Description: The default facility value if the SYSLOG_FACILITY entry does not exist.

default-level()
Type: string
Default: notice

Description: The default level value if the PRIORITY entry does not exist.

host-override()
Type: string
Default:

Description: Replaces the ${HOST} part of the message with the parameter string.

keep-hostname()
Type: yes or no
Default: no

Description: Enable or disable hostname rewriting.

  • If enabled (keep-hostname(yes)), syslog-ng PE will retain the hostname information read from the systemd journal messages.

  • If disabled (keep-hostname(no)), syslog-ng PE will use the hostname that has been set up for the operating system instance that syslog-ng is running on. To query or set this value, use the hostnamectl command.

This option can be specified globally, and per-source as well. The local setting of the source overrides the global option if available.

log-fetch-limit()
Type: number (messages)
Default: 10

Description: The maximum number of messages fetched from a source during a single poll loop. The destination queues might fill up before flow-control could stop reading if log-fetch-limit() is too high.

max-field-size()
Type: number (characters)
Default: 65536

Description: The maximum length of a field's value.

prefix()
Type: string
Default:

Description: If this option is set, every non-built-in mapped names get a prefix (for example: ".SDATA.journald.").

time-zone()
Type: name of the timezone, or the timezone offset
Default:

Description: The default timezone for messages read from the source. Applies only if no timezone is specified within the message itself.

The timezone can be specified as using the name of the (for example time-zone("Europe/Budapest")), or as the timezone offset in +/-HH:MM format (for example +01:00). On Linux and UNIX platforms, the valid timezone names are listed under the /usr/share/zoneinfo directory.

use-fqdn()
Type: yes or no
Default: no

Description: Add Fully Qualified Domain Name instead of short hostname. This option can be specified globally, and per-source as well. The local setting of the source overrides the global option if available.

NOTE:

This option has no effect if the keep-hostname() option is enabled (keep-hostname(yes)) and the message contains a hostname.

Collecting messages from remote hosts using the BSD syslog protocol

NOTE:

The tcp(), tcp6(), udp(), and udp6() drivers are obsolete. Use the network() source and the network() destination instead. For details, see the section called “Collecting messages using the RFC3164 protocol (network() driver)” and the section called “Sending messages to a remote log server using the RFC3164 protocol (network() driver)”, respectively.

The tcp(), tcp6(), udp(), udp6() drivers can receive syslog messages conforming to RFC3164 from the network using the TCP and UDP networking protocols. The tcp6() and udp6() drivers use the IPv6 network protocol, while tcp() and udp() use IPv4.

To convert your existing tcp(), tcp6(), udp(), udp6() source drivers to use the network() driver, see Procedure 6.1, “Change an old source driver to the network() driver”.

tcp(), tcp6(), udp() and udp6() source options — OBSOLETE

NOTE:

The tcp(), tcp6(), udp(), and udp6() drivers are obsolete. Use the network() source and the network() destination instead. For details, see the section called “Collecting messages using the RFC3164 protocol (network() driver)” and the section called “Sending messages to a remote log server using the RFC3164 protocol (network() driver)”, respectively.

To convert your existing tcp(), tcp6(), udp(), udp6() source drivers to use the network() driver, see Procedure 6.1, “Change an old source driver to the network() driver”.

Procedure 6.1. Change an old source driver to the network() driver

To replace your existing tcp(), tcp6(), udp(), udp6() sources with a network() source, complete the following steps.

  1. Replace the driver with network. For example, replace udp( with network(

  2. Set the transport protocol.

    • If you used TLS-encryption, add the transport("tls") option, then continue with the next step.

    • If you used the tcp or tcp6 driver, add the transport("tcp") option.

    • If you used the udp or udp driver, add the transport("udp") option.

  3. If you use IPv6 (that is, the udp6 or tcp6 driver), add the ip-protocol("6") option.

  4. If you did not specify the port used in the old driver, check the section called “network() source options” and verify that your clients send the messages to the default port of the transport protocol you use. Otherwise, set the appropriate port number in your source using the port() option.

  5. All other options are identical. Test your configuration with the syslog-ng --syntax-only command.

    The following configuration shows a simple tcp source.

    source s_old_tcp {
        tcp(
            ip(127.0.0.1) port(1999)
            tls(
                peer-verify("required-trusted")
                key-file("/opt/syslog-ng/etc/syslog-ng/syslog-ng.key")
                cert-file('/opt/syslog-ng/etc/syslog-ng/syslog-ng.crt')
            )
        );
    };

    When replaced with the network() driver, it looks like this.

    source s_new_network_tcp {
        network(
            transport("tls")
            ip(127.0.0.1) port(1999)
            tls(
                peer-verify("required-trusted")
                key-file("/opt/syslog-ng/etc/syslog-ng/syslog-ng.key")
                cert-file('/opt/syslog-ng/etc/syslog-ng/syslog-ng.crt')
            )
        );
    };

Collecting systemd messages using a socket

On platforms running systemd, the systemd-syslog() driver reads the log messages of systemd using the /run/systemd/journal/syslog socket. Note the following points about this driver:

  • If possible, use the more reliable systemd-journal() driver instead.

  • The socket activation of systemd is buggy, causing some log messages to get lost during system startup.

  • If syslog-ng PE is running in a jail or a Linux Container (LXC), it will not read from the /dev/kmsg or /proc/kmsg files.

Declaration: 

systemd-syslog();

Example 6.40. Using the systemd-syslog() driver

@version: 6.0

source s_systemdd {
	systemd-syslog();
};

destination d_network {
	syslog("server.host");
};

log {
	source(s_systemdd);
	destination(d_network);
};

Related Documents