syslog-ng Premium Edition 7.0.14 - Release Notes

Highlights of 7.0.5

XML parser

A new parser, the XML parser has been added, which processes input in XML format, and then adds the parsed data to the message object. Use this parser to interoperate with applications that produce XML-encoded log messages.

The XML parser allows you to extract information from XML logs, and use this information in your logging pipeline, for example, in filters, and also to further process the extracted data using syslog-ng or other tools. In addition, parsing XML logs helps you normalize your log messages, and convert them to a common format.

For details, see "XML parser" in the Administration Guide.

Cisco parser

A new parser, the Cisco parser has been added, which is capable of parsing the log messages of various Cisco devices. The messages of these devices often do not completely comply with the syslog RFCs, making them difficult to parse. The cisco-parser() of syslog-ng PE solves this problem, and can separate these log messages to name-value pairs, extracting also the Cisco-specific values.

For more information, see "Cisco Parser" in the Administration Guide.

New systemd-journal() source option

A new systemd-journal() source option, read-old-records(), has been added. Previously, syslog-ng PE started reading records from the journald system service right from the very beginning of the journal. This was often a lengthy process. The new option lets you specify whether you want to read only new records from the journal or all records, starting from the beginning of the journal.

For more information, see "systemd-journal: Collecting messages from the systemd-journal system log storage" in the Administration Guide.

Configurable JVM options for Java destinations

You can now fine-tune your Java Virtual Machine (JVM) options when configuring Elasticsearch, Hadoop Distributed File System (HDFS), and Apache Kafka destinations. Previously, settings of the Java Virtual Machine could not be overriden from the syslog-ng PE configuration file, resulting sometimes in suboptimal memory utilization. The new jvm-options() allows you to configure these Java settings from syslog-ng PE as a global option.

For details, see:

Changes in HDFS destination options

The following changes have been introduced with regards to HDFS files:

  • New option hdfs-append-enabled(): A new option has been added, which enables syslog-ng PE to append new data to the end of an already existing HDFS file. This means that, when setting this parameter to true, there is no need anymore to open a new file once a file has been closed.

    For further details, see "HDFS destination options" in the Administration Guide.

  • Support for macros in file names and file paths: hdfs-file() now supports the usage of macros, meaning that syslog-ng PE can create files on HDFS dynamically, using macros in the file (or directory) name.

    For further details, see "HDFS destination options" in the Administration Guide.

New TLS options

The following new TLS options have been added:

  • dhparam-file(): Allows you to specify a file that contains the Diffie-Hellman parameters for key exchanges, generated by the openssl dhparam utility.

  • ecdh-curve-list(): Allows you to specify the curves permitted when using Elliptic Curve Cryptography (ECC).

Oracle Linux 6 platform now supported in syslog-ng PE

The Oracle Linux 6 platform is now supported in syslog-ng PE.

For a complete list of supported platforms, see "Supported platforms" in the Administration Guide.

Highlights of 7.0.4

RHEL6 support

Red Hat Enterprise Linux 6 platform is now supported. For details, see "Supported platforms" in the Administration Guide.

New osquery source

The osquery application allows you to ask questions about your machine using an SQL-like language. For example, you can query running processes, logged in users, installed packages and syslog messages as well. You can make queries on demand, and also schedule them to run regularly.

The osquery() source of syslog-ng PE allows you read the results of periodical osquery queries and automatically parse the messages. For details, see "osquery: Collect and parse osquery result logs" in the Administration Guide.

New HTTP destination

The syslog-ng PE application can directly post log messages to web services using the HTTP protocol, without having to use Java.

HTTPS connection, as well as password- and certificate-based authentication is supported. For details, see "http: Posting messages over HTTP without Java" in the Administration Guide.

Look up GeoIP2 data from IP addresses

The syslog-ng PE application can lookup IP addresses from an offline GeoIP2 database, and make the retrieved data available in name-value pairs. Depending on the database used, you can access country code, longitude, and latitude information, and many more in addition to what you could access with geoip(). For details, see "Looking up GeoIP2 data from IP addresses" in the Administration Guide.

You can use the geoip2 template function to format messages to contain GeoIP data. For details, see "Template functions of syslog-ng PE" in the Administration Guide.

The geoip() parser is deprecated.

The geoip() parser is now deprecated. Use the geoip2 parser instead.

Highlights of 7.0.3

Reading Net-SNMP traps

Using the snmptrap() source, you can read and parse the SNMP traps of the Net-SNMP's snmptrapd application. syslog-ng PE can read these traps from a log file, and extract their content into name-value pairs, making it easy to forward them as a structured log message (for example, in JSON format). For details, see "snmptrap: Read Net-SNMP traps" in the Administration Guide.

Monitor syslog-ng PE more effectively

syslog-ng PE version 7.0.3 also includes new metrics. For details, see "Metrics and counters of syslog-ng PE" in the Administration Guide.

Also note the following changes compared earlier syslog-ng PE versions:

  • The stored counter was renamed to queued.

  • The output of the syslog-ng-ctl query command was changed from <counter-name>: <counter-value> to <counter-name>=<counter-value>

Ported from the syslog-ng PE 6 LTS product line
Other changes

Highlights of 7.0.2

Python support: message parsers and template functions

The Python Log Parser allows you to write your own parser in Python. Practically, that way you can process the log message (or parts of the log message) any way you need. For example, you can import external Python modules to process the messages, query databases to enrich the messages with additional data, and many other things. For details, see "Python parser" in the Administration Guide.

You can write your own template function in Python. You can define a Python block in your syslog-ng PE configuration file, and define one or more Python functions in it. You can use these functions as template functions. For details, see the manual pages.

Monitor syslog-ng PE more effectively

The new monitoring() source allows you to granularly select which statistics of syslog-ng PE you want to monitor. In addition, the statistics are available as structured name-value pairs, so you can format the output similarly to other log messages. That way, you can easily convert the statistics and metrics, for example, into JSON or WELF format, and send the results into your monitoring database. For details, see "The monitoring() source" in the Administration Guide.

syslog-ng PE version 7.0.2 also includes the monitoring-welf() source, which is a preconfigured monitoring() source that generates statistics messages in WELF format. Starting with version 7.0., syslog-ng PE uses this driver for new installations to generate statistics (earlier versions use the internal() source for this purpose).

Ported from the syslog-ng PE 6 LTS product line
  • The functionality of syslog-ng-query application is available in syslog-ng PE 7.0.2, as part of the syslog-ng-ctl utility.

Related Documents