Chat now with support
Chat with Support

syslog-ng Premium Edition 7.0.32 - Administration Guide

Preface Introduction to syslog-ng The concepts of syslog-ng Installing syslog-ng PE The syslog-ng PE quick-start guide The syslog-ng PE configuration file Collecting log messages — sources and source drivers
How sources work default-network-drivers: Receive and parse common syslog messages internal: Collecting internal messages file: Collecting messages from text files google-pubsub: collecting messages from the Google Pub/Sub messaging service wildcard-file: Collecting messages from multiple text files linux-audit: Collecting messages from Linux audit logs mssql, oracle, sql: collecting messages from an SQL database network: Collecting messages using the RFC3164 protocol (network() driver) office365: Fetching logs from Office 365 osquery: Collect and parse osquery result logs pipe: Collecting messages from named pipes program: Receiving messages from external applications python: writing server-style Python sources python-fetcher: writing fetcher-style Python sources snmptrap: Read Net-SNMP traps syslog: Collecting messages using the IETF syslog protocol (syslog() driver) system: Collecting the system-specific log messages of a platform systemd-journal: Collecting messages from the systemd-journal system log storage systemd-syslog: Collecting systemd messages using a socket tcp, tcp6,udp, udp6: Collecting messages from remote hosts using the BSD syslog protocol udp-balancer: Receiving UDP messages at very high rate unix-stream, unix-dgram: Collecting messages from UNIX domain sockets windowsevent: Collecting Windows event logs
Sending and storing log messages — destinations and destination drivers
elasticsearch2>: Sending messages directly to Elasticsearch version 2.0 or higher (DEPRECATED) elasticsearch-http: Sending messages to Elasticsearch HTTP Event Collector file: Storing messages in plain-text files google_pubsub(): Sending logs to the Google Cloud Pub/Sub messaging service google_pubsub-managedaccount(): Sending logs to the Google Cloud Pub/Sub messaging service authenticated by Google Cloud managed service account hdfs: Storing messages on the Hadoop Distributed File System (HDFS) http: Posting messages over HTTP kafka(): Publishing messages to Apache Kafka (Java implementation) (DEPRECATED) kafka-c(): Publishing messages to Apache Kafka using the librdkafka client (C implementation) logstore: Storing messages in encrypted files mongodb: Storing messages in a MongoDB database network: Sending messages to a remote log server using the RFC3164 protocol (network() driver) pipe: Sending messages to named pipes program: Sending messages to external applications python: writing custom Python destinations sentinel(): Sending logs to the Microsoft Azure Sentinel cloud snmp: Sending SNMP traps smtp: Generating SMTP messages (email) from logs splunk-hec: Sending messages to Splunk HTTP Event Collector sql(): Storing messages in an SQL database stackdriver: Sending logs to the Google Stackdriver cloud syslog: Sending messages to a remote logserver using the IETF-syslog protocol syslog-ng(): Forward logs to another syslog-ng node tcp, tcp6, udp, udp6: Sending messages to a remote log server using the legacy BSD-syslog protocol (tcp(), udp() drivers) unix-stream, unix-dgram: Sending messages to UNIX domain sockets usertty: Sending messages to a user terminal — usertty() destination Client-side failover
Routing messages: log paths, flags, and filters Global options of syslog-ng PE TLS-encrypted message transfer Advanced Log Transport Protocol Reliability and minimizing the loss of log messages Manipulating messages parser: Parse and segment structured messages Processing message content with a pattern database Correlating log messages Enriching log messages with external data Monitoring statistics and metrics of syslog-ng Multithreading and scaling in syslog-ng PE Troubleshooting syslog-ng Best practices and examples The syslog-ng manual pages Glossary

syslog-ng-ctl.1

Name

syslog-ng-ctl — Display message statistics and enable verbose, debug and trace modes in syslog-ng PE

Synopsis

syslog-ng-ctl [command] [options]

Description

NOTE: The syslog-ng-ctl application is distributed with the syslog-ng PE system logging application, and is usually part of the syslog-ng PE package.

This manual page is only an abstract, for the complete documentation of syslog-ng PE, see the syslog-ng PE Documentation page.

The syslog-ng-ctl application is a utility that can be used to:

  • enable/disable various syslog-ng PE messages for troubleshooting

  • display statistics about the processed messages

  • handling password-protected private keys

  • display the currently running configuration of syslog-ng PE

  • reload the configuration of syslog-ng PE.

  • stop syslog-ng PE.

Enabling troubleshooting messages

command [options]

Use the syslog-ng-ctl <command> --set=on command to display verbose, trace, or debug messages. If you are trying to solve configuration problems, the verbose (and occasionally trace) messages are usually sufficient. Debug messages are needed mostly for finding software errors. After solving the problem, do not forget to turn these messages off using the syslog-ng-ctl <command> --set=off Note that enabling debug messages does not enable verbose and trace messages.

Use syslog-ng-ctl <command> without any parameters to display whether the particular type of messages are enabled or not.

If you need to use a non-standard control socket to access syslog-ng PE, use the syslog-ng-ctl <command> --set=on --control=<socket> command to specify the socket to use.

  • verbose

    Print verbose messages. If syslog-ng PE was started with the --stderr or -e option, the messages will be sent to stderr. If not specified, syslog-ng PE will log such messages to its internal source.

  • trace

    Print trace messages of how messages are processed. If syslog-ng PE was started with the --stderr or -e option, the messages will be sent to stderr. If not specified, syslog-ng PE will log such messages to its internal source.

  • debug

    Print debug messages. If syslog-ng PE was started with the --stderr or -e option, the messages will be sent to stderr. If not specified, syslog-ng PE will log such messages to its internal source.

Example:
syslog-ng-ctl verbose --set=on
syslog-ng-ctl query

The syslog-ng PE application stores various data, metrics, and statistics in a hash table. Every property has a name and a value. For example:

[syslog-ng]
|
|_[destinations]-[network]-[tcp]->[stats]->{received=12;dropped=2}
|
|_[sources]-[sql]-[stats]->{received=501;dropped=0}

You can query the nodes of this tree, and also use filters to select the information you need. A query is actually a path in the tree. You can also use the ? and * wildcards. For example:

  • Select every property: *

  • Select all dropped value from every stats node: *.stats.dropped

The nodes and properties available in the tree depend on your syslog-ng PE configuration (that is, the sources, destinations, and other objects you have configured), and also on your stats-level() settings.

The list command

syslog-ng-ctl query list

Use the syslog-ng-ctl query list command to display the list of metrics that syslog-ng PE collects about the processed messages.

An example output:

center.received.stats.processed
center.queued.stats.processed
destination.java.d_elastic#0.java_dst(ElasticSearch,elasticsearch-syslog-ng-test,t7cde889529c034aea9ec_micek).stats.dropped
destination.java.d_elastic#0.java_dst(ElasticSearch,elasticsearch-syslog-ng-test,t7cde889529c034aea9ec_micek).stats.processed
destination.java.d_elastic#0.java_dst(ElasticSearch,elasticsearch-syslog-ng-test,t7cde889529c034aea9ec_micek).stats.queued
destination.d_elastic.stats.processed
source.s_tcp.stats.processed
source.severity.7.stats.processed
source.severity.0.stats.processed
source.severity.1.stats.processed
source.severity.2.stats.processed
source.severity.3.stats.processed
source.severity.4.stats.processed
source.severity.5.stats.processed
source.severity.6.stats.processed
source.facility.7.stats.processed
source.facility.16.stats.processed
source.facility.8.stats.processed
source.facility.17.stats.processed
source.facility.9.stats.processed
source.facility.18.stats.processed
source.facility.19.stats.processed
source.facility.20.stats.processed
source.facility.0.stats.processed
source.facility.21.stats.processed
source.facility.1.stats.processed
source.facility.10.stats.processed
source.facility.22.stats.processed
source.facility.2.stats.processed
source.facility.11.stats.processed
source.facility.23.stats.processed
source.facility.3.stats.processed
source.facility.12.stats.processed
source.facility.4.stats.processed
source.facility.13.stats.processed
source.facility.5.stats.processed
source.facility.14.stats.processed
source.facility.6.stats.processed
source.facility.15.stats.processed
source.facility.other.stats.processed
global.payload_reallocs.stats.processed
global.msg_clones.stats.processed
global.sdata_updates.stats.processed
tag..source.s_tcp.stats.processed

The syslog-ng-ctl query list command has the following options:

  • --reset

    Use --reset to set the selected counters to 0 after executing the query.

Displaying metrics and statistics

syslog-ng-ctl query get [options]

The syslog-ng-ctl query get <query> command lists the nodes that match the query, and their values.

For example, the "destination*" query lists the configured destinations, and the metrics related to each destination. An example output:

          destination.java.d_elastic#0.java_dst(ElasticSearch,elasticsearch-syslog-ng-test,t7cde889529c034aea9ec_micek).stats.dropped=0
destination.java.d_elastic#0.java_dst(ElasticSearch,elasticsearch-syslog-ng-test,t7cde889529c034aea9ec_micek).stats.processed=0
destination.java.d_elastic#0.java_dst(ElasticSearch,elasticsearch-syslog-ng-test,t7cde889529c034aea9ec_micek).stats.queued=0
destination.d_elastic.stats.processed=0

The syslog-ng-ctl query get command has the following options:

  • --sum

    Add up the result of each matching node and return only a single number.

    For example, the syslog-ng-ctl query get --sum "destination*.dropped" command displays the number of messages dropped by the syslog-ng PE instance.

  • --reset

    Use --reset to set the selected counters to 0 after executing the query.

The stats command

stats [options]

Use the stats command to display statistics about the processed messages. For details about the displayed statistics, see The syslog-ng PE Administrator Guide. The stats command has the following options:

  • --control=<socket> or -c

    Specify the socket to use to access syslog-ng PE. Only needed when using a non-standard socket.

  • --reset=<socket> or -r

    Reset all statistics to zero, except for the queued counters. (The queued counters show the number of messages in the message queue of the destination driver, waiting to be sent to the destination.)

Example:
syslog-ng-ctl stats

An example output:

        src.internal;s_all#0;;a;processed;6445
src.internal;s_all#0;;a;stamp;1268989330
destination;df_auth;;a;processed;404
destination;df_news_dot_notice;;a;processed;0
destination;df_news_dot_err;;a;processed;0
destination;d_ssb;;a;processed;7128
destination;df_uucp;;a;processed;0
source;s_all;;a;processed;7128
destination;df_mail;;a;processed;0
destination;df_user;;a;processed;1
destination;df_daemon;;a;processed;1
destination;df_debug;;a;processed;15
destination;df_messages;;a;processed;54
destination;dp_xconsole;;a;processed;671
dst.tcp;d_network#0;10.50.0.111:514;a;dropped;5080
dst.tcp;d_network#0;10.50.0.111:514;a;processed;7128
dst.tcp;d_network#0;10.50.0.111:514;a;queued;2048
destination;df_syslog;;a;processed;6724
destination;df_facility_dot_warn;;a;processed;0
destination;df_news_dot_crit;;a;processed;0
destination;df_lpr;;a;processed;0
destination;du_all;;a;processed;0
destination;df_facility_dot_info;;a;processed;0
center;;received;a;processed;0
destination;df_kern;;a;processed;70
center;;queued;a;processed;0
destination;df_facility_dot_err;;a;processed;0
Displaying license-related information

syslog-ng-ctl show-license-info [options]

The syslog-ng PE application uses a license in server mode to determine the maximum number of hosts that are allowed to connect. Use the syslog-ng-ctl show-license-info command to display license-related information on the number of hosts currently logging to your server. This helps you to plan your capacity, to check your license usage, and to detect client misconfiguration that can result in a license miscount anomaly. Note that in client or relay mode, syslog-ng PE does not require a license.

The syslog-ng-ctl show-license-info command displays the following information. In case of an unlimited license, or in client or relay mode, only the license type is displayed:

  • License Type: none, limited, unlimited

  • Host Limit: the maximum number of hosts that are allowed to connect.

  • Currently Used Slots: the number of currently used host slots

  • Usage: the percent of used host slots

  • Licensed Clients: the list of hostnames that are stored in the license module

  • Monthly-Consumed-License-Hosts: the current number of used hosts since the previous monthly host usage report. This item appears only if we have a valid and limited license, moreover, the option report-host-usage() is turned on.

The syslog-ng-ctl show-license-info command has the following options:

  • --json or -J

    Print license-related information in JSON format.

Example:
syslog-ng-ctl show-license-info

An example output:

License-Type: limited
Host-Limit: 10
Currently-Used-Slots: 7
Usage: 70%
Licensed-Clients:
    192.168.0.1
    192.168.0.2
    192.168.0.3
    192.168.1.4
    192.168.1.5

Example:

        syslog-ng-ctl show-license-info --json

An example output:

{
    "license_type": "limited",
    "host_limit": 10,
    "currently_used_slots": 7,
    "usage": "70%",
    "licensed_clients": [
        "xy.testdomain",
        "testhost",
        "192.168.0.3",
        "test_host",
        "192.168.1.5"
    ]
}
Example:
syslog-ng-ctl show-license-info

in case of an unlimited license

An example output:

$ syslog-ng-ctl show-license-info
License-Type: unlimited
Example:
syslog-ng-ctl show-license-info

if syslog-ng PE is in client or relay mode

An example output:

$ syslog-ng-ctl show-license-info
License-Type: none
Handling password-protected private keys

syslog-ng-ctl credentials [options]

The syslog-ng-ctl credentials status command allows you to query the status of the private keys that syslog-ng PE uses in the network() and syslog() drivers. You can also provide the passphrase for password-protected private keys using the syslog-ng-ctl credentials add command. For details on using password-protected keys, see The syslog-ng Administrator Guide.

Displaying the status of private keys

syslog-ng-ctl credentials status [options]

The syslog-ng-ctl credentials status command allows you to query the status of the private keys that syslog-ng PE uses in the network() and syslog() drivers. The command returns the list of private keys used, and their status. For example:

syslog-ng-ctl credentials status
Secret store status:
/home/user/ssl_test/client-1/client-encrypted.key SUCCESS

If the status of a key is PENDING, you must provide the passphrase for the key, otherwise syslog-ng PE cannot use it. The sources and destinations that use these keys will not work until you provide the passwords. Other parts of the syslog-ng PE configuration will be unaffected. You must provide the passphrase of the password-protected keys every time syslog-ng PE is restarted.

The following log message also notifies you of PENDING passphrases:

          Waiting for password; keyfile='private.key'

  • --control=<socket> or -c

    Specify the socket to use to access syslog-ng PE. Only needed when using a non-standard socket.

Opening password-protected private keys

syslog-ng-ctl credentials add [options]

You can add the passphrase to a password-protected private key file using the following command. syslog-ng PE will display a prompt for you to enter the passphrase. We recommend that you use this method.

          syslog-ng-ctl credentials add --id=<path-to-the-key>

Alternatively, you can include the passphrase in the --secret parameter:

          syslog-ng-ctl credentials add --id=<path-to-the-key> --secret=<passphrase-of-the-key>

Or you can pipe the passphrase to the syslog-ng-ctl command, for example:

          echo "<passphrase-of-the-key>" | syslog-ng-ctl credentials add --id=<path-to-the-key>

  • --control=<socket> or -c

    Specify the socket to use to access syslog-ng PE. Only needed when using a non-standard socket.

  • --id=<path-to-the-key> or -i

    The path to the password-protected private key file. This is the same path that you use in the key-file() option of the syslog-ng PE configuration file.

  • --secret=<passphrase-of-the-key> or -s

    The password or passphrase of the private key.

Displaying the configuration

syslog-ng-ctl config [options]

Use the syslog-ng-ctl config command to display the configuration that syslog-ng PE is currently running. Note by default, only the content of the main configuration file are displayed, included files are not resolved. To resolve included files and display the entire configuration, use the syslog-ng-ctl config --preprocessed command.

Reloading the configuration

syslog-ng-ctl reload [options]

Use the syslog-ng-ctl reload command to reload the configuration file of syslog-ng PE without having to restart the syslog-ng PE application. The syslog-ng-ctl reload works like a SIGHUP (-1). On Microsoft Windows, this is the only way to reload the configuration of syslog-ng PE.

The syslog-ng-ctl reload command returns 0 if the operation was successful, 1 otherwise.

Stopping syslog-ng PE

syslog-ng-ctl stop [options]

Use the syslog-ng-ctl stop command to stop the syslog-ng PE application The syslog-ng-ctl stop works like a SIGHUP (-15) on Linux and Unix systems. On Microsoft Windows, this is the only way to gracefully stop syslog-ng PE if it is running in the foreground.

Files

/opt/syslog-ng/sbin/syslog-ng-ctl

See also

syslog-ng Documentation page

The syslog-ng.conf manual page

The syslog-ng manual page

NOTE: For the detailed documentation of syslog-ng PE see syslog-ng PE Documentation page.

If you experience any problems or need help with syslog-ng PE, visit the syslog-ng mailing list.

For news and notifications about syslog-ng PE, visit the syslog-ng blogs.

syslog-ng.8

Name

syslog-ng — syslog-ng system logger application

Synopsis

syslog-ng [options]

Description

This manual page is only an abstract, for the complete documentation of syslog-ng PE, see the syslog-ng PE Documentation page.

The syslog-ng PE application is a flexible and highly scalable system logging application. Typically, syslog-ng PE is used to manage log messages and implement centralized logging, where the aim is to collect the log messages of several devices on a single, central log server. The different devices - called syslog-ng clients - all run syslog-ng PE, and collect the log messages from the various applications, files, and other sources. The clients send all important log messages to the remote syslog-ng PE server, where the server sorts and stores them.

Options

  • --caps

    Run syslog-ng PE process with the specified POSIX capability flags.

    • If the --no-caps option is not set, and the host supports CAP_SYSLOG, syslog-ng PE uses the following capabilities: cap_net_bind_service, cap_net_broadcast, cap_net_raw, cap_dac_read_search, cap_dac_override, cap_chown, cap_fowner=p cap_syslog=ep

    • If the --no-caps option is not set, and the host does not support CAP_SYSLOG, syslog-ng PE uses the following capabilities: cap_net_bind_service, cap_net_broadcast, cap_net_raw, cap_dac_read_search, cap_dac_override, cap_chown, cap_fowner=p cap_sys_admin=ep

    For example:

    /opt/syslog-ng/sbin/syslog-ng -Fv --caps cap_sys_admin,cap_chown,cap_dac_override,cap_net_bind_service,cap_fowner=pi

    Note that the capabilities are not case sensitive, the following command is also good: /opt/syslog-ng/sbin/syslog-ng -Fv --caps CAP_SYS_ADMIN,CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_NET_BIND_SERVICE,CAP_FOWNER=pi

    For details on the capability flags, see the following man pages: >cap_from_text(3) and >capabilities(7)

  • --cfgfile <file> or -f <file>

    Use the specified configuration file.

  • --chroot <dir> or -C <dir>

    Change root to the specified directory. The configuration file is read after chrooting so, the configuration file must be available within the chroot. That way it is also possible to reload the syslog-ng configuration after chrooting. However, note that the --user and --groupoptions are resolved before chrooting.

  • --control <file> or -c <file>

    Set the location of the syslog-ng control socket. Default value: >/var/run/syslog-ng.ctl

  • --debug or -d

    Start syslog-ng PE in debug mode.

  • --default-modules

    A comma-separated list of the modules that are loaded automatically. Modules not loaded automatically can be loaded by including the @module <modulename> statement in the syslog-ng PE configuration file. Available only in syslog-ng PE version 4.1 and later.

  • --enable-core

    Enable syslog-ng PE to write core files in case of a crash to help support and debugging.

  • --fd-limit <number>

    Set the minimal number of required file descriptors (fd-s). This sets how many files syslog-ng PE can keep open simultaneously. Default value: 4096. Note that this does not override the global ulimit setting of the host.

  • --foreground or -F

    Do not daemonize, run in the foreground. When running in the foreground, syslog-ng PE starts from the current directory ($CWD) so it can create core files (normally, syslog-ng PE starts from >$PREFIX/var).

  • --group <group> or -g <group>

    Switch to the specified group after initializing the configuration file.

  • --help or -h

    Display a brief help message.

  • --module-registry

    Display the list and description of the available modules. Note that not all of these modules are loaded automatically, only the ones specified in the --default-modules option. Available only in syslog-ng PE 4 F1 and later.

  • --no-caps

    Run syslog-ng PE as root, without capability-support. This is the default behavior. On Linux, it is possible to run syslog-ng PE as non-root with capability-support if syslog-ng PE was compiled with the --enable-linux-caps option enabled. (Run syslog-ng --version to display the list of enabled build parameters.)

    To run syslog-ng PE with specific capabilities, use the --caps option.

  • --persist-file <persist-file> or -R <persist-file>

    Set the path and name of the syslog-ng.persist file where the persistent options and data are stored.

  • --pidfile <pidfile> or -p <pidfile>

    Set path to the PID file where the pid of the main process is stored.

  • --preprocess-into <output-file>

    After processing the configuration file and resolving included files and variables, write the resulting configuration into the specified output file. Available only in syslog-ng PE 4 F1 and later.

  • --process-mode <mode>

    Sets how to run syslog-ng PE: in the foreground (mainly used for debugging), in the background as a daemon, or in safe-background mode. By default, syslog-ng runs in safe-background mode. This mode creates a supervisor process called supervising syslog-ng, that restarts syslog-ng PE if it crashes.

  • --stderr or -e

    Log internal messages of syslog-ng PE to stderr. Mainly used for debugging purposes in conjunction with the --foreground option. If not specified, syslog-ng PE will log such messages to its internal source.

  • --syntax-only or -s

    Verify that the configuration file is syntactically correct and exit.

  • --user <user> or -u <user>

    Switch to the specified user after initializing the configuration file (and optionally chrooting). Note that it is not possible to reload the syslog-ng configuration if the specified user has no privilege to create the >/dev/log file.

  • --verbose or -v

    Enable verbose logging used to troubleshoot syslog-ng PE.

  • --version or -V

    Display version number and compilation information, and also the list and short description of the available modules. For detailed description of the available modules, see the --module-registry option. Note that not all of these modules are loaded automatically, only the ones specified in the --default-modulesoption.

  • --worker-threads

    Sets the number of worker threads syslog-ng PE can use, including the main syslog-ng PE thread. Note that certain operations in syslog-ng PE can use threads that are not limited by this option. This setting has effect only when syslog-ng PE is running in multithreaded mode. Available only in syslog-ng PE 4 F1 and later. See The syslog-ng PE 7 Administrator Guide for details.

Setting default command-line options

You can set default settings for syslog-ng PE — syslog-ng PE will always run with these default command-line parameters. You can specify your default settings in the following files:

  • >/etc/default/syslog-ng

  • >/etc/sysconfig/syslog-ng (only for RedHat platforms)

  • >$SYSLOGNG_PREFIX/etc/default/syslog-ng, where $SYSLOGNG_PREFIX is the installation directory of syslog-ng PE. For version 4.0, this is >/opt/syslog-ng

During startup, syslog-ng PE will automatically use the settings from these files if they exist. You can set the following options:

  • MAXWAIT

    The number of seconds the init script will wait for syslog-ng PE to shut down properly. If the syslog-ng PE process does not shut down during this period, it is terminated with a SIGKILL signal. Increase this value if you have lots of separate disk-buffer files (for example, to 60 seconds).

  • SYSLOGNG_OPTIONS

    A string of additional command-line options for the syslog-ng PE daemon.

Files

>/opt/syslog-ng/

>/opt/syslog-ng/etc/syslog-ng.conf

See also

The syslog-ng.conf manual page

NOTE: For the detailed documentation of syslog-ng PE see syslog-ng PE Documentation page.

If you experience any problems or need help with syslog-ng PE, visit the syslog-ng mailing list.

For news and notifications about syslog-ng PE, visit the syslog-ng blogs.

syslog-ng.conf.5

Name

syslog-ng.conf — syslog-ng configuration file

Synopsis

syslog-ng.conf

Description

This manual page is only an abstract, for the complete documentation of syslog-ng PE, see the syslog-ng PE Documentation page.

The syslog-ng PE application is a flexible and highly scalable system logging application. Typically, syslog-ng PE is used to manage log messages and implement centralized logging, where the aim is to collect the log messages of several devices on a single, central log server. The different devices - called syslog-ng clients - all run syslog-ng PE, and collect the log messages from the various applications, files, and other sources. The clients send all important log messages to the remote syslog-ng PE server, where the server sorts and stores them.

Basic concepts of syslog-ng PE

The syslog-ng PE application reads incoming messages and forwards them to the selected destinations. The syslog-ng PE application can receive messages from files, remote hosts, and other sources.

Log messages enter syslog-ng PE in one of the defined sources, and are sent to one or more destinations.

Sources and destinations are independent objects, log paths define what syslog-ng PE does with a message, connecting the sources to the destinations. A log path consists of one or more sources and one or more destinations: messages arriving from a source are sent to every destination listed in the log path. A log path defined in syslog-ng PE is called a log statement.

Optionally, log paths can include filters. Filters are rules that select only certain messages, for example, selecting only messages sent by a specific application. If a log path includes filters, syslog-ng PE sends only the messages satisfying the filter rules to the destinations set in the log path.

Other optional elements that can appear in log statements are parsers and rewriting rules. Parsers segment messages into different fields to help processing the messages, while rewrite rules modify the messages by adding, replacing, or removing parts of the messages.

Configuring syslog-ng PE

  • The main body of the configuration file consists of object definitions sources, destinations, logpaths define which log message are received and where they are sent. All identifiers, option names and attributes, and any other strings used in the syslog-ng PE configuration file are case sensitive. Object definitions (also called statements) have the following syntax:

    type-of-the-object identifier-of-the-object {<parameters>};

    • Type of the object: One of source, destination, log, filter, parser, rewrite rule, or template.

      Identifier of the object: A unique name identifying the object. When using a reserved word as an identifier, enclose the identifier in quotation marks ("").

      All identifiers, attributes, and any other strings used in the syslog-ng PE configuration file are case sensitive.

      TIP: Use identifiers that refer to the type of the object they identify. For example, prefix source objects with s_, destinations with d_, and so on.

      NOTE:

      Repeating a definition of an object (that is, defining the same object with the same id more than once) is not allowed, unless you use the @define allow-config-dups 1 definition in the configuration file.

    • Parameters: The parameters of the object, enclosed in braces {parameters}.

    • Semicolon: Object definitions end with a semicolon (;).

    For example, the following line defines a source and calls it s_internal.

    source s_internal { internal(); };

    The object can be later referenced in other statements using its ID, for example, the previous source is used as a parameter of the following log statement:

    log { source(s_internal); destination(d_file); };

  • The parameters and options within a statement are similar to function calls of the C programming language: the name of the option followed by a list of its parameters enclosed within brackets and terminated with a semicolon.

    option(parameter1, parameter2); option2(parameter1, parameter2);

    For example, the file() driver in the following source statement has three options: the filename (/var/log/apache/access.log), follow-freq(), and flags(). The follow-freq() option also has a parameter, while the flags() option has two parameters.

    source s_tail { file("/var/log/apache/access.log"
    follow-freq(1) flags(no-parse, validate-utf8)); };

    Objects may have required and optional parameters. Required parameters are positional, meaning that they must be specified in a defined order. Optional parameters can be specified in any order using the option(value) format. If a parameter (optional or required) is not specified, its default value is used. The parameters and their default values are listed in the reference section of the particular object.

    Example: Using required and optional parameters

    The unix-stream() source driver has a single required argument: the name of the socket to listen on. Optional parameters follow the socket name in any order, so the following source definitions have the same effect:

    source s_demo_stream1 {
        unix-stream("<path-to-socket>" max-connections(10) group(log)); };
source s_demo_stream2 {
        unix-stream("<path-to-socket>" group(log) max-connections(10)); };

  • Some options are global options, or can be set globally, for example, whether syslog-ng PE should use DNS resolution to resolve IP addresses.

    options { use-dns(no); };

  • Objects can be used before definition.

  • Objects can be defined inline as well. This is useful if you use the object only once (for example, a filter).

  • To add comments to the configuration file, start a line with # and write your comments. These lines are ignored by syslog-ng PE.

                  # Comment: This is a stream source
source s_demo_stream {
        unix-stream("<path-to-socket>" max-connections(10) group(log)); };

The syntax of log statements is as follows:

log {
    source(s1); source(s2); ...
    optional_element(filter1|parser1|rewrite1);
    optional_element(filter2|parser2|rewrite2);
    ...
    destination(d1); destination(d2); ...
    flags(flag1[, flag2...]);
};

The following log statement sends all messages arriving to the localhost to a remote server.

source s_localhost { network(ip(127.0.0.1) port(1999)); };
destination d_tcp { network("10.1.2.3" port(1999) localport(999)); };
log { source(s_localhost); destination(d_tcp); };

The syslog-ng PE application has a number of global options governing DNS usage, the timestamp format used, and other general points. Each option may have parameters, similarly to driver specifications. To set global options add an option statement to the syslog-ng PE configuration file using the following syntax:

        options { option1(params); option2(params); ... };
Example: Using global options

To disable domain name resolving, add the following line to the syslog-ng PE configuration file:

options { use-dns(no); };

The sources, destinations, and filters available in syslog-ng PE are listed below. For details, see the syslog-ng PE Documentation page.

Table: Source drivers available in syslog-ng PE
Name Description
file() Opens the specified file and reads messages.
internal() Messages generated internally in syslog-ng PE.
network() Receives messages from remote hosts using the BSD-syslog protocol over IPv4 and IPv6. Supports the TCP, UDP, and TLS network protocols.
pipe() Opens the specified named pipe and reads messages.
program() Opens the specified application and reads messages from its standard output.
sun-stream(), sun-streams() Opens the specified STREAMS device on Solaris systems and reads incoming messages.
syslog() Listens for incoming messages using the new IETF-standard syslog protocol.
system() Automatically detects which platform syslog-ng PE is running on, and collects the native log messages of that platform.
systemd-journal() Collects messages directly from the journal of platforms that use systemd.
systemd-syslog() Collects messages from the journal using a socket on platforms that use systemd.
unix-dgram() Opens the specified unix socket in SOCK_DGRAM mode and listens for incoming messages.
unix-stream() Opens the specified unix socket in SOCK_STREAM mode and listens for incoming messages.
windowsevent() Reads messages from the Windows Event Collector tool.
Table: Destination drivers available in syslog-ng PE
Name Description
elasticsearch2 Sends messages to an Elasticsearch server. The elasticsearch2 driver supports Elasticsearch version 2 and newer.
file() Writes messages to the specified file.
hdfs() Sends messages into a file on a Hadoop Distributed File System (HDFS) or MapR-FS node.
http() Sends messages over the HTTP protocol.
kafka() Publishes log messages to the Apache Kafka message bus, where subscribers can access them.
logstore() Writes messages securely into encrypted, compressed, and timestamped binary files.
mongodb() Sends messages to a MongoDB database.
network() Sends messages to a remote host using the BSD-syslog protocol over IPv4 and IPv6. Supports the TCP, UDP, and TLS network protocols.
pipe() Writes messages to the specified named pipe.
program() Forks and launches the specified program, and sends messages to its standard input.
smtp() Sends email messages to the specified recipients.
Splunk Forward your log messages to Splunk.
sql() Sends messages into an SQL database. In addition to the standard syslog-ng packages, the sql() destination requires database-specific packages to be installed.
syslog() Sends messages to the specified remote host using the IETF-syslog protocol. The IETF standard supports message transport using the UDP, TCP, and TLS networking protocols.
unix-dgram() Sends messages to the specified unix socket in SOCK_DGRAM style (BSD).
unix-stream() Sends messages to the specified unix socket in SOCK_STREAM style (Linux).
usertty() Sends messages to the terminal of the specified user, if the user is logged in.
Table: Filter functions available in syslog-ng PE
Name Description
facility() Filter messages based on the sending facility.
filter() Call another filter function.
host() Filter messages based on the sending host.
inlist() File-based whitelisting and blacklisting.
level() or priority() Filter messages based on their priority.
match() Use a regular expression to filter messages based on a specified header or content field.
message() Use a regular expression to filter messages based on their content.
netmask() Filter messages based on the IP address of the sending host.
program() Filter messages based on the sending application.
source() Select messages of the specified syslog-ng PE source statement.
tags() Select messages having the specified tag.
Files

/opt/syslog-ng/

/opt/syslog-ng/etc/syslog-ng.conf

See also

The syslog-ng manual page

NOTE: For the detailed documentation of syslog-ng PE see syslog-ng PE Documentation page.

If you experience any problems or need help with syslog-ng PE, visit the syslog-ng mailing list.

For news and notifications about syslog-ng PE, visit the syslog-ng blogs.

Glossary

alias IP
auditing policy
authentication
BOM
BSD-syslog protocol
CA
Cadence
certificate
client mode
destination
destination driver
destination, local
destination, network
disk buffer
disk queue
domain name
Drop-down
embedded log statement
filter
fully qualified domain name (FQDN)
gateway
Glossary
high availability
host
hostname
IETF-syslog protocol
key pair
license
log path
log source host
log statement
logstore
Long Term Supported release
LSH
name server
Note
Oracle Instant Client
output buffer
output queue
overflow queue
parser
ping
port
Public-key authentication
regular expression
relay mode
rewrite rule
SaaS
server mode
Skin
Snippet
source
source driver
source, local
source, network
SSL
syslog-ng
syslog-ng agent
syslog-ng client
syslog-ng Premium Edition
syslog-ng relay
syslog-ng server
template
Tip
TLS
traceroute
UNIX domain socket
Related Documents

The document was helpful.

Select Rating

I easily found the information I needed.

Select Rating