Chat now with support
Chat with Support

syslog-ng Premium Edition 7.0.9 - Administration Guide

Preface Introduction to syslog-ng The concepts of syslog-ng Installing syslog-ng The syslog-ng PE quick-start guide The syslog-ng PE configuration file Collecting log messages — sources and source drivers
How sources work default-network-drivers: Receive and parse common syslog messages internal: Collecting internal messages file: Collecting messages from text files wildcard-file: Collecting messages from multiple text files network: Collecting messages using the RFC3164 protocol (network() driver) osquery: Collect and parse osquery result logs pipe: Collecting messages from named pipes program: Receiving messages from external applications snmptrap: Read Net-SNMP traps sun-streams: Collecting messages on Sun Solaris syslog: Collecting messages using the IETF syslog protocol (syslog() driver) system: Collecting the system-specific log messages of a platform systemd-journal: Collecting messages from the systemd-journal system log storage systemd-syslog: Collecting systemd messages using a socket tcp, tcp6, udp, udp6: Collecting messages from remote hosts using the BSD syslog protocol unix-stream, unix-dgram: Collecting messages from UNIX domain sockets windowsevent: Collecting Windows event logs
Sending and storing log messages — destinations and destination drivers
elasticsearch: Sending messages directly to Elasticsearch version 1.x elasticsearch2: Sending messages directly to Elasticsearch version 2.0 or higher file: Storing messages in plain-text files hdfs: Storing messages on the Hadoop Distributed File System (HDFS) http: Posting messages over HTTP kafka: Publishing messages to Apache Kafka logstore: Storing messages in encrypted files mongodb: Storing messages in a MongoDB database network: Sending messages to a remote log server using the RFC3164 protocol (network() driver) pipe: Sending messages to named pipes program: Sending messages to external applications smtp: Generating SMTP messages (e-mail) from logs Splunk: Sending log messages to Splunk sql: Storing messages in an SQL database syslog: Sending messages to a remote logserver using the IETF-syslog protocol syslog-ng: Forwarding messages and tags to another syslog-ng node tcp, tcp6, udp, udp6: Sending messages to a remote log server using the legacy BSD-syslog protocol (tcp(), udp() drivers) unix-stream, unix-dgram: Sending messages to UNIX domain sockets usertty: Sending messages to a user terminal — usertty() destination
Routing messages: log paths, flags, and filters Global options of syslog-ng PE TLS-encrypted message transfer Reliable Log Transfer Protocol Manipulating messages Parsers and segmenting structured messages Processing message content with a pattern database Correlating log messages Enriching log messages with external data Monitoring statistics and metrics of syslog-ng Multithreading and scaling in syslog-ng PE Troubleshooting syslog-ng Best practices and examples The syslog-ng manual pages About us

Installing syslog-ng PE in server mode

Purpose:

Complete the following steps to install syslog-ng PE on log servers. For details on the different operation modes of syslog-ng PE, see Modes of operation.

Steps:
  1. Login to the Support Portal and download the syslog-ng PE installer package and your syslog-ng Premium Edition license file (license.txt). The license will be required to run syslog-ng PE in server mode (see Server mode) and is needed when you are installing syslog-ng PE on your central log server.

  2. Enable the executable attribute for the installer using the chmod +x syslog-ng-<edition>-<version>-<OS>-<platform>.run, then start the installer as root using the ./syslog-ng-<edition>-<version>-<OS>-<platform>.run command. (Note that the exact name of the file depends on the operating system and platform.) Wait until the package is uncompressed and the welcome screen appears, then select Continue.

    Figure 12: The welcome screen

  3. Accepting the EULA: You can install syslog-ng PE only if you understand and accept the terms of the End-User License Agreement (EULA). The full text of the EULA can be displayed during installation by selecting the Show EULA option, and is also available in this guide for convenience at Software Transaction, License and End User License Agreements. Select Accept to accept the EULA and continue the installation.

    If you do not accept the terms of the EULA for some reason, select Reject to cancel installing syslog-ng PE.

  4. Detecting platform and operating system: The installer attempts to automatically detect your oprating system and platform. If the displayed information is correct, select Yes. Otherwise select Exit to abort the installation, and verify that your platform is supported. For a list of supported platforms, see Supported platforms. If your platform is supported but not detected correctly, contact your local distributor, reseller, or access the Support Portal. For contact details, see About us.

    Figure 13: Platform detection

  5. Installation path: Enter the path to install syslog-ng PE to. This is useful if you intend to install syslog-ng PE without registering it as a service, or if it cannot be installed to the default location because of policy compliance reasons. If no path is given, syslog-ng PE is installed to the default folder.

    Figure 14: Installation path

    NOTE:

    When installing syslog-ng PE to an alternative path on AIX, HP-UX, or Solaris platforms, set the CHARSETALIASDIR environmental variable to the lib subdirectory of the installation path. That way syslog-ng PE can find the charset.alias file.

  6. Registering as syslog service: Select Register to register syslog-ng PE as the syslog service. This will stop and disable the default syslog service of the system.

    Figure 15: Registering as syslog service

  7. Locating the license: Enter the path to your license file (license.txt) and select OK. Typically this is required only for your central log server.

    If you are upgrading an existing configuration that already has a license file, the installer automatically detects it.

    Figure 16: Platform detection

  8. Upgrading: The syslog-ng PE installer can automatically detect if you have previously installed a version of syslog-ng PE on your system. To use the configuration file of this previous installation, select Yes. To ignore the old configuration file and create a new one, select No.

    Note that if you decide to use your existing configuration file, the installer automatically checks it for syntax error and displays a list of warnings and errors if it finds any problems.

    Figure 17: Upgrading syslog-ng

  9. Generating a new configuration file: The installer displays some questions to generate a new configuration file.

    1. Remote sources: Select Yes to accept log messages from the network. TCP, UDP, and SYSLOG messages on every interface will be automatically accepted.

      Figure 18: Accepting remote messages

    2. Remote destinations: Enter the IP address or hostname of your log server or relay and select OK.

      Figure 19: Forwarding messages to the log server

    NOTE:

    Accepting remote messages and forwarding them to a log server means that syslog-ng PE will start in relay mode.

  10. After the installation is finished, add the /opt/syslog-ng/bin and /opt/syslog-ng/sbin directories to your search PATH environment variable. That way you can use syslog-ng PE and its related tools without having to specify the full pathname. Add the following line to your shell profile:

    PATH=/opt/syslog-ng/bin:$PATH 

    NOTE:

    The native logrotation tools do not send a SIGHUP to syslog-ng after rotating the log files, causing syslog-ng to write into files already rotated. To solve this problem, the syslog-ng init script links the /var/run/syslog.pid file to syslog-ng's pid. Also, on Linux, the install.sh script symlinks the initscript of the original syslog daemon to syslog-ng's initscript.

  11. (Optional step for SELinux-enabled systems): Complete Using syslog-ng PE on SELinux.

Installing syslog-ng PE without user-interaction

The syslog-ng PE application can be installed in silent mode without any user-interaction by specifying the required parameters from the command line. Answers to every question of the installer can be set in advance using command-line parameters.

./syslog-ng-premium-edition-<version>.run -- --silent [options]

Caution:

The -- characters between the executable and the parameters are mandatory, like in the following example: ./syslog-ng-premium-edition-3.0.1b-solaris-10-sparc-client.run -- --silent --accept-eula -l /var/tmp/license.txt

To display the list of parameters, execute the ./syslog-ng-premium-edition-<version>.run -- --h command. Currently the following options are available:

  • --accept-eula or -a: Accept the EULA.

  • --license-file <file> or -l <file>: Path to the license file.

  • --upgrade | -u: Perform automatic upgrade — use the configuration file from an existing installation.

  • --remote <destination host>: Send logs to the specified remote server. Not available when performing an upgrade.

  • --network: Accept messages from the network. Not available when performing an upgrade.

  • --configuration <file>: Use the specified configuration file.

  • --list-installed: List information about all installed syslog-ngs.

  • --path <path>: Set installation path.

  • --register: Force service registration.

  • --no-register: Prevent service registration.

Installing syslog-ng PE on RPM-based platforms (Red Hat, SUSE, AIX)

Purpose:

To install syslog-ng PE on operating systems that use the Red Hat Package Manager (RPM), complete the following steps. Installing syslog-ng PE automatically replaces the original syslog service. The following supported operating systems use RPM:

  • Red Hat Enterprise Linux

  • Red Hat Enterprise Server

  • SUSE Linux Enterprise Server

Caution:

If you already had syslog-ng Open Source Edition (OSE) installed on the host, and are upgrading to syslog-ng Premium Edition, make sure that the ${SYSLOGNG_OPTIONS} environmental variable does not contain a -p <path-to-pid-file> option. If it does, remove this option from the environmental variable, because it can prevent syslog-ng PE from stopping properly. Typically, the environmental variable is set in the files /etc/default/syslog-ng or /etc/sysconfig/syslog-ng, depending on the operating system you use.

Caution:

If you are planning to use Python in syslog-ng PE (for example Python parser or Python template function) on RHEL 6 platform, then you have to manually install Python 2.7. If the Python version on the machine is not 2.7, you will receive a similar error message during startup:

[2017-07-27T13:42:03.606679] Reading shared object for a candidate module; path='/opt/syslog-ng/lib/syslog-ng', fname='mod-python.so', module='mod-python' [2017-07-27T13:42:03.606994] Error opening plugin module; module='mod-python', error='libpython2.7.so.1.0: cannot open shared object file: No such file or directory'

Steps:
  1. Login to the Support Portal and download the syslog-ng RPM package for your system.

    • If the host already uses syslog-ng PE for logging, execute the following command as root. Otherwise, skip this step.

      rpm -U syslog-ng-premium-edition-<version>-<OS>-<arch>.rpm

      The syslog-ng Premium Edition application and all its dependencies will be installed, and the configuration of the existing syslog-ng PE installation will be used.

      NOTE:

      If you are upgrading from syslog-ng version 2.1, note that the location of the configuration file has been moved to /opt/syslog-ng/etc/syslog-ng.conf

    • Execute the following command as root:

      rpm -i syslog-ng-premium-edition-<version>-<OS>-<arch>.rpm

      The syslog-ng PE application and all its dependencies will be installed.

  2. Caution:

    When performing an upgrade, the package manager might automatically execute the post-uninstall script of the upgraded package, stopping syslog-ng PE and starting syslogd. If this happens, stop syslogd and start syslog-ng PE by issuing the following commands:

    /etc/init.d/syslogd stop
    /etc/init.d/syslog-ng start

    This behavior has been detected on CentOS 4 systems, but may occur on other rpm-based platforms as well.

  3. Edit the syslog-ng PE configuration file as needed. If you want to run syslog-ng PE in server mode, copy the license file to the /opt/syslog-ng/etc/ directory.

    For information on configuring syslog-ng PE, see the The syslog-ng PE quick-start guide.

  4. (Optional step for SELinux-enabled systems): Complete Using syslog-ng PE on SELinux.

Using syslog-ng PE on SELinux

Purpose:

Version syslog-ng PE 5 F2 and later properly supports SELinux on Red Hat Enterprise Linux 6.5 and newer platforms. Version 5 F5 and later also supports SELinux on Red Hat Enterprise Linux 5, as well as on 6.0-6.4. The CentOS and Oracle Linux platforms corresponding to the supported RHEL versions are supported as well. To use syslog-ng PE on a SELinux-enabled host, complete the following steps.

NOTE:

The following steps install SELinux policy module that enables syslog-ng PE to properly run with its default configuration and default installation path (/opt/syslog-ng) on a SELinux-enabled host. If you configure syslog-ng PE to perform an operation that is outside the permissions of this policy module (for example, to bind to a non-standard port, use a program destination or source, or to write logfiles in a non-standard directory), you have to modify and recompile the policy module. If you need help with that, access the Support Portal. For contact details, see About us.

Prerequisites:
  • The following packages must be available on the host: policycoreutils,policycoreutils-devel, policycoreutils-python. If they are not already installed, issue the following command: yum install policycoreutils policycoreutils-devel policycoreutils-python

  • On RHEL 6.5, update the following packages at least to the indicated versions. These packages are available in the Red Hat repositories and are installed by default on RHEL 6.6. You can update them with the yum update selinux-policy command.

    • selinux-policy-3.7.19-231.el6.noarch > 3.7.19-260.el6.noarch

    • selinux-policy-targeted-3.7.19-231.el6.noarch > 3.7.19-260.el6.noarch

  • The syslog-ng PE application must be installed on the host. For details, see Installing syslog-ng.

Steps:
Expected result:

The syslog-ng PE application is installed and properly running under SELinux. If syslog-ng PE does not start, or displays permission errors, execute the syslog_ng.sh.

  1. Download the SELinux Policy Module from the syslog-ng PE SELinux policy module from syslog-ng Premium Edition - Download Software page.

  2. Uncompress the file and run the ./syslog_ng.sh script to compile and load the SELinux rules for syslog-ng PE.

  3. Restart syslog-ng PE using the following command.

    Caution:

    The SELinux policy works only if syslog-ng PE is started by the init daemon.

    • On RHEL6: service syslog-ng restart

    • On RHEL7: systemctl restart syslog-ng

    If you do not use the service or the systemctl to start syslog-ng PE execute the syslog_ng.sh script again after starting syslog-ng PE. This is required to correct the settings of the files related to syslog-ng PE (most notably /dev/log and the files under /opt/syslog-ng). The settings can become incorrect if the privileges of the process that started syslog-ng PE are different from the privileges of the service or the systemctl process.

  4. (Optional): The syslog-ng PE application can create coredumps, but this is disabled by default. You can enable coredumps with the setsebool -P daemons_dump_core 1 command.

    Note that his command enables every daemons on your system to create core dumps, not just syslog-ng PE. There is no way to enable per-application core dumps in SELinux.

Related Documents

The document was helpful.

Select Rating

I easily found the information I needed.

Select Rating