Chat now with support
Chat mit Support

syslog-ng Premium Edition 7.0.18 - Administration Guide

Preface Introduction to syslog-ng The concepts of syslog-ng Installing syslog-ng The syslog-ng PE quick-start guide The syslog-ng PE configuration file Collecting log messages — sources and source drivers
How sources work default-network-drivers: Receive and parse common syslog messages internal: Collecting internal messages file: Collecting messages from text files wildcard-file: Collecting messages from multiple text files linux-audit: Collecting messages from Linux audit logs network: Collecting messages using the RFC3164 protocol (network() driver) office365: Fetching logs from Office 365 osquery: Collect and parse osquery result logs pipe: Collecting messages from named pipes program: Receiving messages from external applications python: writing server-style Python sources python-fetcher: writing fetcher-style Python sources snmptrap: Read Net-SNMP traps sun-streams: Collecting messages on Sun Solaris syslog: Collecting messages using the IETF syslog protocol (syslog() driver) system: Collecting the system-specific log messages of a platform systemd-journal: Collecting messages from the systemd-journal system log storage systemd-syslog: Collecting systemd messages using a socket tcp, tcp6, udp, udp6: Collecting messages from remote hosts using the BSD syslog protocol udp-balancer: Receiving UDP messages at very high rate unix-stream, unix-dgram: Collecting messages from UNIX domain sockets windowsevent: Collecting Windows event logs
Sending and storing log messages — destinations and destination drivers
elasticsearch2: Sending messages directly to Elasticsearch version 2.0 or higher (DEPRECATED) elasticsearch-http: Sending messages to Elasticsearch HTTP Event Collector file: Storing messages in plain-text files hdfs: Storing messages on the Hadoop Distributed File System (HDFS) http: Posting messages over HTTP kafka: Publishing messages to Apache Kafka logstore: Storing messages in encrypted files mongodb: Storing messages in a MongoDB database network: Sending messages to a remote log server using the RFC3164 protocol (network() driver) pipe: Sending messages to named pipes program: Sending messages to external applications python: writing custom Python destinations smtp: Generating SMTP messages (email) from logs splunk-hec: Sending messages to Splunk HTTP Event Collector sql: Storing messages in an SQL database stackdriver: Sending logs to the Google Stackdriver cloud syslog: Sending messages to a remote logserver using the IETF-syslog protocol syslog-ng(): Forward logs to another syslog-ng node tcp, tcp6, udp, udp6: Sending messages to a remote log server using the legacy BSD-syslog protocol (tcp(), udp() drivers) unix-stream, unix-dgram: Sending messages to UNIX domain sockets usertty: Sending messages to a user terminal — usertty() destination Client-side failover
Routing messages: log paths, flags, and filters Global options of syslog-ng PE TLS-encrypted message transfer Advanced Log Transfer Protocol Reliability and minimizing the loss of log messages Manipulating messages parser: Parse and segment structured messages Processing message content with a pattern database Correlating log messages Enriching log messages with external data Monitoring statistics and metrics of syslog-ng Multithreading and scaling in syslog-ng PE Troubleshooting syslog-ng Best practices and examples The syslog-ng manual pages Glossary

How relaying log messages works

Depending on your exact needs about relaying log messages, there are many scenarios and syslog-ng PE options that influence how the log message will look like on the logserver. Some of the most common cases are summarized in the following example:

Consider the following example: client-host > syslog-ng-relay > syslog-ng-server, where the IP address of client-host is 192.168.1.2. The client-host device sends a syslog message to syslog-ng-relay. Depending on the settings of syslog-ng-relay, the following can happen.

  • By default, the keep-hostname() option is disabled, so syslog-ng-relay writes the IP address of the sender host (in this case, 192.168.1.2) to the HOST field of the syslog message, discarding any IP address or hostname that was originally in the message.

  • If the keep-hostname() option is enabled on syslog-ng-relay, but name resolution is disabled (the use-dns() option is set to no), syslog-ng-relay uses the HOST field of the message as-is, which is probably 192.168.1.2.

  • To resolve the 192.168.1.2 IP address to a hostname on syslog-ng-relay using a DNS server, use the keep-hostname(no) and use-dns(yes) options. If the DNS server is properly configured and reverse DNS lookup is available for the 192.168.1.2 address, syslog-ng PE will rewrite the HOST field of the log message to client-host.

    NOTE:

    It is also possible to resolve IP addresses locally, without relying on the DNS server. For details on local name resolution, see Resolving hostnames locally.

  • The above points apply to the syslog-ng PE server (syslog-ng-server) as well, so if syslog-ng-relay is configured properly, use the keep-hostname(yes) option on syslog-ng-server to retain the proper HOST field. Setting keep-hostname(no) on syslog-ng-server would result in syslog-ng PE rewriting the HOST field to the address of the host that sent the message to syslog-ng-server, which is syslog-ng-relay in this case.

  • If you cannot or do not want to resolve the 192.168.1.2 IP address on syslog-ng-relay, but want to store your log messages on syslog-ng-server using the IP address of the original host (that is, client-host), you can enable the spoof-source() option on syslog-ng-relay. However, spoof-source() works only under the following conditions:

    • The syslog-ng PE binary has been compiled with the --enable-spoof-source option.

    • The log messages are sent using the highly unreliable UDP transport protocol. (Extremely unrecommended.)

Managing and checking syslog-ng PE service on Linux

This section describes how to start, stop and check the status of syslog-ng Premium Edition (syslog-ng PE) service on Linux.

Starting syslog-ng PE

To start syslog-ng PE, execute the following command as root.

Example: starting syslog-ng PE

systemctl start syslog-ng

If the service starts successfully, no output will be displayed.

The following message indicates that syslog-ng PE can not start (see Checking syslog-ng PE status):

Job for syslog-ng.service failed because the control process exited with error code. See systemctl status syslog-ng.service and journalctl -xe for details.

Stopping syslog-ng PE

To stop syslog-ng PE

  1. Execute the following command as root.
    Example: command for stopping syslog-ng PE

    systemctl stop syslog-ng

  2. Check the status of syslog-ng PE service (see Checking syslog-ng PE status).

Restarting syslog-ng PE

To restart syslog-ng PE, execute the following command as root.

Example: command for restarting syslog-ng PE

systemctl restart syslog-ng

Reloading configuration file without restarting syslog-ng PE

To reload the configuration file without restarting syslog-ng PE, execute the following command as root.

Example: command for reloading the configuration file without restarting syslog-ng PE

systemctl reload syslog-ng

Checking syslog-ng PE status

To check the following status-related components, observe the suggestions below.

  • Checking the status of syslog-ng PE service

    To check the status of syslog-ng PE service

    1. Execute the following command as root.

      Example: command for checking the status of syslog-ng PE service

      systemctl --no-pager status syslog-ng

    2. Check the Active: field, which shows the status of syslog-ng PE service. The following statuses are possible:

      • active (running) - syslog-ng PE service is up and running

        Example: syslog-ng PE service active
        syslog-ng.service - System Logger Daemon
Loaded: loaded (/lib/systemd/system/syslog-ng.service; enabled; vendor preset: enabled)
Active: active (running) since Tue 2019-06-25 08:58:09 CEST; 5s ago
Main PID: 6575 (syslog-ng)
Tasks: 3
Memory: 13.3M
CPU: 268ms
CGroup: /system.slice/syslog-ng.service
6575 /opt/syslog-ng/libexec/syslog-ng -F --no-caps --enable-core

      • inactive (dead) - syslog-ng service is stopped

        Example: syslog-ng PE status inactive
        syslog-ng.service - System Logger Daemon
Loaded: loaded (/lib/systemd/system/syslog-ng.service; enabled; vendor preset: enabled)
Active: inactive (dead) since Tue 2019-06-25 09:14:16 CEST; 2min 18s ago
Process: 6575 ExecStart=/opt/syslog-ng/sbin/syslog-ng -F --no-caps --enable-core $SYSLOGNG_OPTIONS (code=exited, status=0/SUCCESS)
Main PID: 6575 (code=exited, status=0/SUCCESS)
Status: "Shutting down... Tue Jun 25 09:14:16 2019"
Jun 25 09:14:31 as-syslog-srv systemd: Stopped System Logger Daemon.

  • Checking the process of syslog-ng PE

    To check the process of syslog-ng PE, execute one of the following commands.

    • Example: command ps u `pidof syslog-ng`

      ps u `pidof syslog-ng`

      Expected output example:

      USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND

      syslogng 6709 0.0 0.6 308680 13432 ? Ss 09:17 0:00 /opt/syslog-ng/libexec/syslog-ng -F --no-caps --enable-core

    • Example: command ps axu | grep syslog-ng | grep -v grep

      ps axu | grep syslog-ng | grep -v grep

      Expected output example:

      syslogng 6709 0.0 0.6 308680 13432 ? Ss 09:17 0:00 /opt/syslog-ng/libexec/syslog-ng -F --no-caps --enable-core

  • Checking the internal logs of syslog-ng PE

    The internal logs of syslog-ng PE contains informal, warning and error messages.

    By default, syslog-ng PE log messages (generated on the internal() source) are written to /var/log/messages.

    Check the internal logs of syslog-ng PE for any issue.

  • Message processing

    The syslog-ng PE application collects statistics about the number of processed messages on the different sources and destinations.

    • Central statistics

      To check the central statistics, execute the following command to see the number of received and queued (sent) messages by syslog-ng PE.

      Example: command for checking central statistics

      watch "/opt/syslog-ng/sbin/syslog-ng-ctl stats | grep ^center"

      The output will be updated in every 2 seconds.

      If the numbers are changing, syslog-ng PE is processing the messages.

      Example: output example 
      Every 2.0s: /opt/syslog-ng/sbin/syslog-ng-ctl stats | grep ^center       Tue Jun 25 10:33:25 2019
center;;queued;a;processed;112
center;;received;a;processed;28

    • Source statistics

      To check the source statistics, execute the following command to see the number of received messages on the configured sources.

      Example: command for checking central statistics

      watch "/opt/syslog-ng/sbin/syslog-ng-ctl stats | grep ^source"

      The output will be updated in every 2 seconds.

      If the numbers are changing, syslog-ng PE is receiving messages on the sources.

      Example: output example 
      Every 2.0s: /opt/syslog-ng/sbin/syslog-ng-ctl stats | grep ^source      Tue Jun 25 10:40:50 2019
source;s_null;;a;processed;0
source;s_net;;a;processed;0
source;s_local;;a;processed;90

    • Destination statistics

      To check the source statistics, execute the following command to see the number of received messages on the configured sources.

      Example: command for checking destination statistics

      watch "/opt/syslog-ng/sbin/syslog-ng-ctl stats | grep ^source"

      The output will be updated in every 2 seconds.

      If the numbers are changing, syslog-ng PE is receiving messages on the sources.

      Example: output example 
      Every 2.0s: /opt/syslog-ng/sbin/syslog-ng-ctl stats | grep ^destination      Tue Jun 25 10:41:02 2019
destination;d_logserver2;;a;processed;90
destination;d_messages;;a;processed;180
destination;d_logserver;;a;processed;90
destination;d_null;;a;processed;0

NOTE:If you find error messages in the internal logs, messages are not processed by syslog-ng PE or you encounter any issue, you have the following options:

The syslog-ng PE configuration file

Location of the syslog-ng configuration file

The syslog-ng application is configured by editing the syslog-ng.conf file. Use any regular text editor application to modify the file. The syslog-ng.conf and license.txt files are located under the /opt/syslog-ng/etc/ directory.

Verwandte Dokumente