Batch support in the http() destination driver

The http() destination can now send a batch of log messages in a single HTTP request, greatly improving the performance. In addition, this feature also allows you to post proper JSON-encoded arrays as POST payloads, which is required by several REST APIs. For details, see Administration Guide.

Write your own destination in Python

Extending syslog-ng OSE in Python has been supported for several releases, but so far this feature was mostly undocumented. Now you can find more details about this feature in "python: writing custom Python destinations" in the Administration Guide.

Write your own message source in Python

Starting with syslog-ng OSE version 3.18, you can write custom message sources in Python. Both server-style and fetcher-style sources are supported. For more details, see "python: writing server-style Python sources" in the Administration Guide and "python-fetcher: writing fetcher-style Python sources" in the Administration Guide.

Enhancements

• When hdfs-append-enabled is set to true, syslog-ng OSE will append new data to the end of an already existing HDFS file. Note that in this case, archiving is automatically disabled, and syslog-ng OSE will ignore the hdfs-archive-dir option.

• The hdfs destination now supports the time-reap() option.

• The urlencode() template function has been renamed to url-encode(). Also, the telegram() destination now automatically encodes the messages.

• New template functions are available: url-decode() and base64-encode(). For details, see "Template functions of syslog-ng OSE" in the Administration Guide.

• The syslog-ng-ctl config command can display the contents of the configuration file that syslog-ng OSE is currently running.

• The rekey option of value-pairs() now supports a new transformation: shift-levels. It cuts dot-delimited "levels" in the name (including the initial dot). For example, --shift-levels 2 deletes the prefix up to the second dot in the name of the key: .iptables.SRC becomes SRC

  For details, see "value-pairs()" in the Administration Guide.

• The value-pairs() option now has a new scope: none. This scope resets previously added scopes, making it possible to get remove automatically added name-value pairs from the scope.

  For details, see "value-pairs()" in the Administration Guide.

• When receiving messages with the default-network-drivers() source, syslog-ng OSE now automatically sets the ${.app.name} name-value pair to the name of the application that sent the log message.

  For details, see "default-network-drivers: Receive and parse common syslog messages" in the Administration Guide.

Deprecated features

The elasticsearch() destination has been deprecated, because it supports only ElasticSearch version 1.x, which has been End-of-Life since January, 2017. Use the elasticsearch2() destination instead.