Chat now with support
Chat with Support

syslog-ng Open Source Edition 3.18 - Administration Guide

Preface Introduction to syslog-ng The concepts of syslog-ng Installing syslog-ng The syslog-ng OSE quick-start guide The syslog-ng OSE configuration file source: Read, receive, and collect log messages
How sources work default-network-drivers: Receive and parse common syslog messages internal: Collecting internal messages file: Collecting messages from text files wildcard-file: Collecting messages from multiple text files linux-audit: Collecting messages from Linux audit logs network: Collecting messages using the RFC3164 protocol (network() driver) nodejs: Receiving JSON messages from nodejs applications mbox: Converting local e-mail messages to log messages osquery: Collect and parse osquery result logs pipe: Collecting messages from named pipes pacct: Collecting process accounting logs on Linux program: Receiving messages from external applications python: writing server-style Python sources python-fetcher: writing fetcher-style Python sources snmptrap: Read Net-SNMP traps sun-streams: Collecting messages on Sun Solaris syslog: Collecting messages using the IETF syslog protocol (syslog() driver) system: Collecting the system-specific log messages of a platform systemd-journal: Collecting messages from the systemd-journal system log storage systemd-syslog: Collecting systemd messages using a socket tcp, tcp6, udp, udp6: Collecting messages from remote hosts using the BSD syslog protocol— OBSOLETE unix-stream, unix-dgram: Collecting messages from UNIX domain sockets stdin: Collecting messages from the standard input stream
destination: Forward, send, and store log messages
amqp: Publishing messages using AMQP elasticsearch: Sending messages directly to Elasticsearch version 1.x (DEPRECATED) elasticsearch2: Sending logs directly to Elasticsearch and Kibana 2.0 or higher file: Storing messages in plain-text files graphite: Sending metrics to Graphite Sending logs to Graylog hdfs: Storing messages on the Hadoop Distributed File System (HDFS) Posting messages over HTTP http: Posting messages over HTTP without Java kafka: Publishing messages to Apache Kafka loggly: Using Loggly logmatic: Using Logmatic.io mongodb: Storing messages in a MongoDB database network: Sending messages to a remote log server using the RFC3164 protocol (network() driver) osquery: Sending log messages to osquery's syslog table pipe: Sending messages to named pipes program: Sending messages to external applications pseudofile() python: writing custom Python destinations redis: Storing name-value pairs in Redis riemann: Monitoring your data with Riemann smtp: Generating SMTP messages (e-mail) from logs Splunk: Sending log messages to Splunk sql: Storing messages in an SQL database stomp: Publishing messages using STOMP syslog: Sending messages to a remote logserver using the IETF-syslog protocol syslog-ng: Forwarding messages and tags to another syslog-ng node tcp, tcp6, udp, udp6: Sending messages to a remote log server using the legacy BSD-syslog protocol (tcp(), udp() drivers) Telegram: Sending messages to Telegram unix-stream, unix-dgram: Sending messages to UNIX domain sockets usertty: Sending messages to a user terminal: usertty() destination Write your own custom destination in Java or Python Client-side failover
log: Filter and route log messages using log paths, flags, and filters Global options of syslog-ng OSE TLS-encrypted message transfer template and rewrite: Format, modify, and manipulate log messages parser: Parse and segment structured messages db-parser: Process message content with a pattern database (patterndb) Correlating log messages Enriching log messages with external data Statistics of syslog-ng Multithreading and scaling in syslog-ng OSE Troubleshooting syslog-ng Best practices and examples The syslog-ng manual pages Third-party contributions Creative Commons Attribution Non-commercial No Derivatives (by-nc-nd) License About us

Batch support in the http() destination driver

Version 3.18 of syslog-ng Open Source Edition includes the following main features.

The http() destination can now send a batch of log messages in a single HTTP request, greatly improving the performance. In addition, this feature also allows you to post proper JSON-encoded arrays as POST payloads, which is required by several REST APIs. For details, see Administration Guide.

Write your own destination in Python

Extending syslog-ng OSE in Python has been supported for several releases, but so far this feature was mostly undocumented. Now you can find more details about this feature in "python: writing custom Python destinations" in the Administration Guide.

Write your own message source in Python

Starting with syslog-ng OSE version 3.18, you can write custom message sources in Python. Both server-style and fetcher-style sources are supported. For more details, see "python: writing server-style Python sources" in the Administration Guide and "python-fetcher: writing fetcher-style Python sources" in the Administration Guide.

Enhancements

  • When hdfs-append-enabled is set to true, syslog-ng OSE will append new data to the end of an already existing HDFS file. Note that in this case, archiving is automatically disabled, and syslog-ng OSE will ignore the hdfs-archive-dir option.

  • The hdfs destination now supports the time-reap() option.

  • The urlencode() template function has been renamed to url-encode(). Also, the telegram() destination now automatically encodes the messages.

  • New template functions are available: url-decode() and base64-encode(). For details, see "Template functions of syslog-ng OSE" in the Administration Guide.

  • The syslog-ng-ctl config command can display the contents of the configuration file that syslog-ng OSE is currently running.

  • The rekey option of value-pairs() now supports a new transformation: shift-levels. It cuts dot-delimited "levels" in the name (including the initial dot). For example, --shift-levels 2 deletes the prefix up to the second dot in the name of the key: .iptables.SRC becomes SRC

    For details, see "value-pairs()" in the Administration Guide.

  • The value-pairs() option now has a new scope: none. This scope resets previously added scopes, making it possible to get remove automatically added name-value pairs from the scope.

    For details, see "value-pairs()" in the Administration Guide.

  • When receiving messages with the default-network-drivers() source, syslog-ng OSE now automatically sets the ${.app.name} name-value pair to the name of the application that sent the log message.

    For details, see "default-network-drivers: Receive and parse common syslog messages" in the Administration Guide.

Deprecated features

The elasticsearch() destination has been deprecated, because it supports only ElasticSearch version 1.x, which has been End-of-Life since January, 2017. Use the elasticsearch2() destination instead.

Related Documents

The document was helpful.

Select Rating

I easily found the information I needed.

Select Rating