Release Notes
07 February 2023, 12:55
These release notes provide information about the syslog-ng Premium Edition release. For the most recent documents and product information, see syslog-ng Premium Edition - Technical Documentation.
Welcome to syslog-ng Premium Edition (syslog-ng PE) version 6 LTS and thank you for choosing our product. This document describes the new features and most important changes since the latest release of syslog-ng PE. The main aim of this paper is to aid system administrators in planning the migration to the new version of syslog-ng PE. The following sections describe the news and highlights of syslog-ng PE 6 LTS.
This document covers the 6 LTS (long-term supported) release of the syslog-ng Premium Edition product.
The following release policy applies to syslog-ng Premium Edition:
Long Term Supported or LTS releases (for example, syslog-ng Agent 4 LTS) are supported for 3 years after their original publication date and for 1 year after the next LTS release is published (whichever date is later). The second digit of the revisions of such releases is 0 (for example, syslog-ng PE 4.0.1). Maintenance releases to LTS releases contain only bugfixes and security updates.
Feature releases (for example, syslog-ng Agent 4 F1) are supported for 6 months after their original publication date and for 2 months after succeeding Feature or LTS Release is published (whichever date is later). Feature releases contain enhancements and new features, presumably 1-3 new feature per release. Only the last of the feature releases is supported (for example when a new feature release comes out, the last one becomes unsupported).
For a full description on stable and feature releases, see the Version policy.
|
|
Caution:
Downgrading from a feature release to an earlier (and thus unsupported) feature release, or to the previous LTS release is officially not supported, but usually works as long as your syslog-ng PE configuration file is appropriate for the old syslog-ng PE version. However, persistent data like the position of the last processed message in a file source will be probably lost. Logstore files created with a newer version of syslog-ng PE might not be readable with an older version of syslog-ng PE. |
syslog-ng PE 6 LTS is a patch release, with no new functionality. See Resolved issues.
The syslog-ng PE application can separate a message consisting of whitespace or comma-separated key=value pairs (for example, Postfix log messages) into name-value pairs. You can also specify other separator character instead of the equal sign, for example, colon (:) to parse MySQL log messages.
The OpenSSL application used in syslog-ng PE has been updated to version 1.0.2, so you can use new, stronger ciphers to protect the communication between your syslog-ng PE clients and servers.
Also, you can now specify the curves that are permitted in the connection using the list-curves() option, and also import Diffie-Hellman parameters from a file using the dhparam-file() option.
Starting from version 6 LTS, the syslog-ng PE installer packages (earlier called client and server package) are renamed. From now on, the package with the compact suffix does not include SQL support. If you do not use the sql() source or destination on the host, use the compact binaries. That way no unnecessary components are installed to your system. Earlier, the names of packages containing sql() support included the server suffix, this suffix has been removed from the package names.
The graphical interface of the syslog-ng Agent for Windows application now supports version 4.0 of the Microsoft .NET Framework.
Group Policies can be configured directly from the syslog-ng Agent for Windows interface.
When creating a filter based on Event Source names, the Windows Agent interface displayed a list of possible sources. However, the names of this list did not always match the actual name of the Event Source (as displayed in the Event Viewer). From now on, the Windows Agent does not list the possible Event Source names. To create a filter using Event Source names, first find the "Source:" field as shown on the General tab of the Event Viewer, and enter its value in the filter.
New platforms in syslog-ng PE 6 LTS
Ubuntu 16.04 LTS (Xenial Xerus)
Platforms not supported in syslog-ng PE 6 LTS
AIX 6.1
FreeBSD 9
HP-UX 11i v2
Debian 7 (wheezy) and Ubuntu 12.04 LTS (Precise Pangolin) is supported only until version 6.0.15, and unsupported in 6.0.16 and later versions.
New platforms in syslog-ng Agent for Windows 6 LTS
Windows 10
Windows Server 2012
Windows Server 2016 (in syslog-ng PE version 6.0.4)
Windows Server 2019 (in syslog-ng PE version 6.0.16)
Platforms not supported in syslog-ng Agent for Windows 6 LTS
Microsoft Windows Server 2003
Microsoft Windows XP SP3
Version 5 F6 of syslog-ng PE supports Elasticsearch version 2.0 and newer. Because of compatibility reasons, syslog-ng PE has a separate destination (elasticsearch2()) that you can use with Elasticsearch version 2.0 and newer.
X-Pack security (Elasticsearch Shield) is supported for both Elasticsearch 1.x and 2.x destinations, allowing you to authenticate your syslog-ng PE clients on the Elasticsearch server.
Version 5.4 of syslog-ng PE can directly send log messages to Elasticsearch, allowing you to search and analyze your data in real time, and visualize it with Kibana.
The syslog-ng PE application allows you to publish your log data to your Apache Kafka message bus, where subscribers can access them, making it easy to integrate your log data into a big data solution.
Version 5.3 of syslog-ng PE can send plain-text log files to the Hadoop Distributed File System (HDFS), allowing you to store your log data on a distributed, scalable file system. This is especially useful if you have huge amount of log messages that would be difficult to store otherwise, or if you want to process your messages using Hadoop tools (for example, Apache Pig).
MongoDB is a schema-free, document-oriented database, ideal to collect log messages. Since it does not require a predetermined schema, it gives you much more flexibility than the SQL databases, making it it easy to store name-value pairs extracted from log messages. That way, you can conveniently store metadata received in a log message (for example, in the SDATA part of RFC5424-formatted log messages), or other data parsed from the body of the log messages (for example, usernames parsed from login/logout messages).
An important aspect of logging is alerting on important but rare events. A common way to do that is via e-mail, sent either to people or to services which process them further. The SMTP destination driver allows you to send e-mails based on incoming log messages to one or more addresses.
JavaScript Object Notation (JSON) is a text-based open standard designed for human-readable data interchange. It is used primarily to transmit data between a server and web application, serving as an alternative to XML. The syslog-ng PE application can separate parts of JSON-encoded log messages to name-value pairs, allowing you to receive structured log messages from such applications, store them, and convert them to other format if needed.
The systemd journal is a new type of system log storage. This is used, or will be used on most Linux distributions, such as RHEL (from RHEL7), Fedora, CentOS, and so on. The journal can store name-value pairs instead of the traditional system log entries. syslog-ng PE 6 LTS can directly read log messages from the journal file of platforms using systemd.
syslog-ng PE version 5 F6 includes a new template function (format-cef-extension) to format name-value pairs as ArcSight Common Event Format extensions. Note that the template function only formats the selected name-value pairs, it does not provide any mapping. There is no special support for creating the prefix part of a Common Event Format message.
The syslog-ng PE application can convert messages or selected value-pairs into JavaScript Object Notation (JSON) format. Including the template function in a message template allows you to store selected information about a log message (that is, its content, macros, or other metadata) in JSON format, or to forward JSON messages to external applications.
The syslog-ng PE application allows you to select and construct name-value pairs from any information already available about the log message, or extracted from the message itself. You can directly use this structured information, for example, in the following places:
mongodb() destination
format-welf() template function
or in other destinations using the format-json() template function.
Using the groupset rewrite rule, you can rewrite multiple macros at the same time, making it easy to modify the values of fields parsed using patterndb or from JSON. You can specify the list of macros manually, or also as a glob pattern.
The syslog-ng PE application can compare the value of a macro to a list of strings. Earlier, if a specific macro had several different values, filtering on the macro values required several filter statements. Now you can specify every expected value in a file, and use the contents of that file in a filter.
With the hash template function, you can create message digests from parts of the log message.
HOSTID is a 32-bit number generated by a cryptographically secure pseudorandom number generator. Its purpose is to identify the syslog-ng PE host, thus it is the same for every message of the host. It can be accessed via the $HOSTID macro (which cannot be rewritten).
When the global option use-uniqid(yes) is set, syslog-ng PE generates this practically unique id for every received or locally generated message. You can add this ID to your messages using the $UNIQID macro.
The syslog-ng PE application uses a license in server mode to determine the maximum number of hosts that are allowed to connect. Use the syslog-ng-ctl show-license-info command to display license-related information the number of hosts currently logging to your server. This helps you to plan your capacity, to check your license usage, and to detect client misconfiguration that can result in a license miscount anomaly. Note that in client or relay mode, syslog-ng PE does not require a license.
To simplify the management of large-scale syslog-ng PE deployments, you can now centrally manage your syslog-ng PE hosts from Puppet. The syslog-ng Premium Edition Puppet module allows you to perform the following tasks.
Install syslog-ng PE from a package repository.
Upgrade syslog-ng PE to a newer version.
Delete syslog-ng PE from a host.
Update the syslog-ng PE configuration file of your hosts from a central repository.
Create backup of your syslog-ng PE configuration files. You can redistribute these backups to your hosts if a rollback is needed.
The Puppet module supports the following platforms: Red Hat Enterprise Linux (RHEL), Oracle Linux, CentOS, Ubuntu, and Debian. Other Linux platforms based on .deb and .rpm packages might also work, but are not tested.
So far, you could access statistics only in unstructured format, using the syslog-ng-ctl stats command. Now you can query information from a running syslog-ng PE instance using the new syslog-ng-query utility. This tool allows you to access selected statistics in a controlled way, making it easy to process or monitor the results. This is a first step in a new statistics framework that aims to improve the how syslog-ng PE instances can be monitored.
Note that this new framework might decrease the performance of syslog-ng PE under very high load. If you experience any issues, contact the Balabit Support Team and let us know the details of your use case, so we can correct the problem.
In addition to Red Hat Enterprise Linux 6.5, syslog-ng PE now supports SELinux on Red Hat Enterprise Linux 5, as well as on 6.0-6.4. The CentOS platforms corresponding to the supported RHEL versions are supported as well.
You can now specify the location where syslog-ng PE stores the disk-buffer files using the dir() option of disk-buffer(). Note that the dir() option overrides the settings of the --qdisk-dir command-line option.
Value-pairs now have a new option to select every value-pair that has a name beginning with a specified prefix, but remove the prefix when formatting the message.
The syslog-ng Agent on Windows application now searches for CA certificates also in the Intermediate Certification Authorities Store.
So far, the largest message syslog-ng PE could handle was 64kbyte, because internally syslog-ng PE represented the messages in a 16-bit nvtable. From now on, syslog-ng PE uses a 32-bit nvtable, allowing you to receive and manipulate messages much larger than 64kbyte.
The sql() source and destination driver on the HP-UX platform does not support the Oracle SQL database anymore.
On other platforms where Oracle SQL database is supported, the sql() source and destination driver supports version 12 of the Oracle SQL database.
CSV-parsers can use strings as delimiters. For details, see Section delimiters().
Multithreading is enabled by default (threaded(yes)).
The persist-tool utility has a new --rename-key option to help solve troubleshooting the persist file.
The syslog-ng-ctl utility has a new stats --reset option that resets all statistics counters to zero.
A new utility called syslog-debun is available in syslog-ng PE 6 LTS. This tool can be used to collect information about your syslog-ng PE environment into a debug bundle to simplify troubleshooting and increase the speed of solving support tickets.
When using the program() destination, the external application keeps on running when syslog-ng PE exits if the keep-alive() option is set.
So far, you could create custom configuration blocks that had a fixed number of arguments. You can now create custom configuration blocks that can receive variable number of arguments, making the configuration of syslog-ng PE even more flexible. For example, this can be useful when passing arguments to a template, or optional arguments to an underlying driver.
© 2024 One Identity LLC. ALL RIGHTS RESERVED. Terms of Use Privacy Cookie Preference Center
