syslog-ng Premium Edition 7.0.29 - Administration Guide

Version 7.0.22 - 7.0.23

Version 7.0.23 of syslog-ng PE includes the following main features.

• Clustering support for Windows Event Collector (WEC)

  For more information, see "WEC clustering in syslog-ng PE" in the Windows Event Collector Administration Guide.

• Support for new platforms:

  • Debian 10 (Debian Buster)

  • Ubuntu 20.04 LTS (Focal Fossa)

  For more information, see Supported platforms.

• Support for Proxy Protocol

  For more information, see Proxy Protocol support.

Version 7.0.21 - 7.0.22

Version 7.0.22 of syslog-ng PE includes the following main features.

• Collecting messages from Google Pub/Sub with the google-pubsub() source.

  For more information, see google-pubsub: collecting messages from the Google Pub/Sub messaging service.

• More robust upgrade from syslog-ng PE 6LTS

  NOTE: The upgrade from syslog-ng PE 6LTS has been made more robust starting with this version. If you are upgrading from syslog-ng PE 6LTS, first upgrade to the latest available syslog-ng PE 6LTS release, then upgrade to syslog-ng PE 7LTS.

Version 7.0.20 - 7.0.21

Version 7.0.21 of syslog-ng PE includes the following main features.

• Collecting messages from SQL databases with mssql(), oracle(), or sql() sources.

  For more information, see mssql, oracle, sql: collecting messages from an SQL database.

• Sending logs to the Google Cloud Pub/Sub messaging service with the google_pubsub() destination.

  For more information, see google_pubsub(): Sending logs to the Google Cloud Pub/Sub messaging service.

• You can set the value of the fetch-no-data-delay() option of the office365() source to fractions.

  For more information, see office365() source options.

• Fraction second truncation: when the frac-digits() option is set to a value higher than 6, syslog-ng PE will truncate the fraction seconds in the timestamps after 6 digits.

  For more information, see frac-digits().

Version 7.0.19 - 7.0.20

Version 7.0.20 of syslog-ng PE includes the following main features.

• Red Hat EL 8 support

  Starting with version 7.0.20, syslog-ng PE supports Red Hat EL 8 (kernel version 4.18).

  For more information, see Supported platforms and udp-balancer: Receiving UDP messages at very high rate.

• SNMP destination

  The snmp() destination of syslog-ng PE can send log messages to an SNMP destination.

  For more information, see snmp: Sending SNMP traps.

• The syslog-ng PE application uses its own Python interpreter

  Starting from version 7.0.20, the syslog-ng PE application uses its own Python interpreter (shipped with the default syslog-ng PE installation) instead of the system's Python interpreter.

  NOTE: The syslog-ng PE application's built-in Python interpreter only supports Python 3. As a result, any custom Python code used in syslog-ng PE must be compatible with Python 3.

• Disabling and forcing TLSv1.3 in TLS contexts

  • The tlsv1_3 value has been added to the available values of ssl-version() for drivers based on the http() destination.

  • The no-tlsv13 value has been added to the available values of ssl-options() in TLS contexts.

• New statistics item: the connections counter

  The connections counter of syslog-ng PE displays the number of active connections for network-based sources in the syslog-ng-ctl stats statistics information.

  For more information, see connections statistics counter.

Version 7.0.18 - 7.0.19

Version 7.0.19 of syslog-ng PE includes the following main features.

• Sending logs to Microsoft Azure Sentinel

  Version 7.0.19 of syslog-ng PE can directly post log messages to the Microsoft Azure Sentinel cloud using Microsoft Azure Sentinel's public HTTP Data Collector API interface.

  For more information, see sentinel(): Sending logs to the Microsoft Azure Sentinel cloud.

• Persist name assigned to Python sources and destinations

  NOTE: Starting with 7.0.193.0.26, syslog-ng PE assigns a persist name to Python sources and destinations. The persist name is generated from the class name. If you want to use the same Python class multiple times in your syslog-ng PE configuration, add a unique persist-name() to each source or destination, otherwise syslog-ng PE will not start. For example:

  log {
      source { python(class(PyNetworkSource) options("port" "8080") persist-name("<unique-string>); };
      source { python(class(PyNetworkSource) options("port" "8081")); };
    };

  Alternatively, you can include the following line in the Python package: @staticmethod generate_persist_name. For example:

  from syslogng import LogSource
    class PyNetworSource(LogSource):
      @staticmethod
      def generate_persist_name(options):
          return options["port"]
      def run(self):
          pass
      def request_exit(self):
          pass

• OpenSSL version 1.1.1d

  From version 7.0.19, syslog-ng PE supports OpenSSL 1.1.1d for Linux glibc 2.11 and Ubuntu Bionic.

  For more information, see Prerequisites to installing syslog-ng PE.

• Changes in documentation

  • The mongodb() destination options section has been updated.

    The following deprecated options have been removed from the mongodb() destination:

    • database()
    • password()
    • path()
    • safe-mode()
    • servers()
    • username()

  • New information has been added to the documentation about emptying disk-buffer() files and about getting information about disk-buffer() files.

    For more information, see How to empty disk-buffer files and How to get information about disk-buffer files.

Version 7.0.14 - 7.0.16

Changes in product

• The Office 365 support is not experimental anymore. For more information, see "office365: Fetching logs from Office 365" in the Administration Guide.

Changes in documentation

• Managing and checking syslog-ng PE service on Linux has been added to the document.

Version 7.0.14 - 7.0.16

Changes in product

• The XML parser uses the list-handling functionality to handle lists in the XML. For more information, see "Limitations of the XML parsers" in the Administration Guide.

• The list-* template functions allow you to manipulate comma-separated lists. For more information, see List manipulation.

Changes in documentation

• You can use a relay for many different use cases. For more information, see Relay mode.

• Detailed documentation about upgrading from syslog-ng PE 6 to version 7 is available at Upgrading from syslog-ng PE 6.0.x to version 7.

• Detailed documentation about upgrading from syslog-ng PE 6 to version 7 is available at Upgrading from syslog-ng PE 7.0.x to version 7.

Version 7.0.13 - 7.0.14

Changes in product

• syslog-ng PE can send log messages to the Google Stackdriver cloud. Google Stackdriver is a widely used metrics, event, and log aggregator and analyzer system. For details, see "stackdriver: Sending logs to the Google Stackdriver cloud" in the Administration Guide.

• Version 7.0.143.21 of syslog-ng PE can directly post log messages to an Elasticsearch deployment using the Elasticsearch Bulk API over the HTTP and Secure HTTP (HTTPS) protocols. For details, see "elasticsearch-http: Sending messages to Elasticsearch HTTP Event Collector" in the Administration Guide.

Version 7.0.12 - 7.0.13

Changes in product

• The Linux Audit Parser can parse the log messages of the Linux Audit subsystem (auditd). The syslog-ng PE application can separate these log messages to name-value pairs. For details, see "Linux audit parser" in the Administration Guide.

• The windowsevent() source can now automatically process XML arrays, making the array elements available as name-value pairs. For details, see "windowsevent: Collecting Windows event logs" in the Administration Guide.

• Installing the syslog-ng Premium Edition application in Docker containers is now officially supported on CentOS 7, Red Hat EL 7.5, and Ubuntu 18.04 (Bionic Beaver) platforms. For details, see "Installing syslog-ng in Docker" in the Administration Guide.

• The persist-tool utility is now part of the syslog-ng PE package. For details, see the persist-tool manual page.

• Since ElasticSearch version 1.x has reached its end of life, its support has been removed from syslog-ng PE. Use the elasticsearch2 destination instead.

Version 7.0.11 - 7.0.12

Changes in product

• Version 7.0.12 of syslog-ng PE can directly post log messages to a Splunk deployment using the HTTP Event Collector (HEC) over the HTTP and Secure HTTP (HTTPS) protocols. The solution is optimized for performance, and supports sending messages in batch mode, multithreaded message sending, and load-balancing to multiple Splunk indexer nodes.

  HTTPS connection, as well as password- and certificate-based authentication is supported. The content of the events is sent in JSON format.

  For details, see "splunk-hec: Sending messages to Splunk HTTP Event Collector" in the Administration Guide.

• The http() destination now supports load balancing, so a single syslog-ng PE instance can feed log data to multiple HTTP servers, for example, multiple ingestion nodes of an Elasticsearch cluster. For details, see "Batch mode and load balancing" in the Administration Guide.

  HTTP and HTTPS redirections now also handled automatically.

• The syslog() and network() drivers now support the so-reuseport() option that allows multiple sockets on the same host to bind to the same port, improving the performance of multithreaded network server applications running on top of multicore systems.

• The Cisco parser now supports Cisco Catalyst formatted triplets.

• Version 7.0.12 of syslog-ng PE is now available on the Ubuntu 18.04 platform. Note that the Java-based drivers of syslog-ng PE (used for Apache Kafka, Elasticsearch, HDFS) require Java 8, Java 10 is not supported.

• The allow-compress() option of the ALTP communication has been renamed to allow-plain-compress().

Version 7.0.10 - 7.0.11

Changes in product

• Starting with syslog-ng PE version 3.187.0.11, you can write custom message sources in Python. Both server-style and fetcher-style sources are supported. For more details, see "python: writing server-style Python sources" in the Administration Guide and "python-fetcher: writing fetcher-style Python sources" in the Administration Guide.

• You can now configure syslog-ng PE to reset the counter that stores the list of known hosts. That way, you can make syslog-ng PE forget old clients that do not exist anymore, and otherwise would be counted against the license limit. This is especially useful in large datacenters or cloud environments where the client hosts are deployed and removed frequently.

  For details, see the "Global options" in the Administration Guide.

• When hdfs-append-enabled is set to true, syslog-ng PE will append new data to the end of an already existing HDFS file. Note that in this case, archiving is automatically disabled, and syslog-ng PE will ignore the hdfs-archive-dir option.

• New template functions are available: url-decode(), url-encode() and base64-encode(). For details, see "Template functions of syslog-ng PE" in the Administration Guide.

• The syslog-ng-ctl config command can display the contents of the configuration file that syslog-ng PE is currently running.

Changes in documentation

Extending syslog-ng PE in Python has been supported for several releases, but so far this feature was mostly undocumented. Now you can find more details about this feature in "python: writing custom Python destinations" in the Administration Guide.

Version 7.0.9 - 7.0.10

Changes in product

• It is now possible to not only receive but also send messages using ALTP to hosts that are running version 6 or at least version 7.0.9 of syslog-ng PE or the syslog-ng Agent for Windows application. For details, see Advanced Log Transfer Protocol .
• The global flush-lines() option does not affect the batch size of ALTP any more. Instead, a new ALTP has been introduced, called batch-size(). For details, see ALTP options.

• The failover() option allows you to specify what happens after syslog-ng PE fails over to a secondary server. Additionally, the failover-servers() option has been deprecated and removed from the document. For more information about the failover() option, see Client-side failover.

• You can now refer to any additional parameters at the end of the argument in a block by adding three dots to it (…). It tells syslog-ng PE that this macro accepts `__VARARGS__`, therefore any name-value pair can be passed without validation. For details, see Passing arguments to configuration blocks.

• You can now make parameters mandatory in block definitions by defining them with empty brackets (). For details, see Mandatory parameters.