Chat now with support
Chat with Support

syslog-ng Premium Edition 7.0.9 - Administration Guide

Preface Introduction to syslog-ng The concepts of syslog-ng Installing syslog-ng The syslog-ng PE quick-start guide The syslog-ng PE configuration file Collecting log messages — sources and source drivers
How sources work default-network-drivers: Receive and parse common syslog messages internal: Collecting internal messages file: Collecting messages from text files wildcard-file: Collecting messages from multiple text files network: Collecting messages using the RFC3164 protocol (network() driver) osquery: Collect and parse osquery result logs pipe: Collecting messages from named pipes program: Receiving messages from external applications snmptrap: Read Net-SNMP traps sun-streams: Collecting messages on Sun Solaris syslog: Collecting messages using the IETF syslog protocol (syslog() driver) system: Collecting the system-specific log messages of a platform systemd-journal: Collecting messages from the systemd-journal system log storage systemd-syslog: Collecting systemd messages using a socket tcp, tcp6, udp, udp6: Collecting messages from remote hosts using the BSD syslog protocol unix-stream, unix-dgram: Collecting messages from UNIX domain sockets windowsevent: Collecting Windows event logs
Sending and storing log messages — destinations and destination drivers
elasticsearch: Sending messages directly to Elasticsearch version 1.x elasticsearch2: Sending messages directly to Elasticsearch version 2.0 or higher file: Storing messages in plain-text files hdfs: Storing messages on the Hadoop Distributed File System (HDFS) http: Posting messages over HTTP kafka: Publishing messages to Apache Kafka logstore: Storing messages in encrypted files mongodb: Storing messages in a MongoDB database network: Sending messages to a remote log server using the RFC3164 protocol (network() driver) pipe: Sending messages to named pipes program: Sending messages to external applications smtp: Generating SMTP messages (e-mail) from logs Splunk: Sending log messages to Splunk sql: Storing messages in an SQL database syslog: Sending messages to a remote logserver using the IETF-syslog protocol syslog-ng: Forwarding messages and tags to another syslog-ng node tcp, tcp6, udp, udp6: Sending messages to a remote log server using the legacy BSD-syslog protocol (tcp(), udp() drivers) unix-stream, unix-dgram: Sending messages to UNIX domain sockets usertty: Sending messages to a user terminal — usertty() destination
Routing messages: log paths, flags, and filters Global options of syslog-ng PE TLS-encrypted message transfer Reliable Log Transfer Protocol Manipulating messages Parsers and segmenting structured messages Processing message content with a pattern database Correlating log messages Enriching log messages with external data Monitoring statistics and metrics of syslog-ng Multithreading and scaling in syslog-ng PE Troubleshooting syslog-ng Best practices and examples The syslog-ng manual pages About us

RLTP options

The following options are specific to the ALTP protocol.

Caution:

Version 7.0.9 of syslog-ng Premium Edition can receive messages using Advanced Log Transfer Protocol (it is supported only as a source), sending messages is currently not implemented.

Note that in earlier versions of syslog-ng Premium Edition, Advanced Log Transfer Protocol is called Reliable Log Transfer Protocol (RLTP).

allow-compress()
Accepted values: yes | no
Default: no

Description: Enable on-the-wire compression in the ALTP communication. Note that this option must be enabled both on the server and the client side to have any effect. Enabling compression can significantly reduce the bandwidth required to transport the messages, but can slightly decrease the performance of syslog-ng PE, reducing the number of transferred messages. The allow-compress() option can be used in source and destination drivers as well. Available in syslog-ng PE 5.0 and later.

message-acknowledgement-timeout()
Type: number (seconds)
Default: 900

Description: When the receiver (syslog-ng PE server) receives and successfully processes a message, it sends an acknowledgement to the sender (the syslog-ng PE client). If the receiver does not acknowledge receiving the messages within this period, the sender terminates the connection with the receiver. Use this option only in destination drivers.

response-timeout()
Type: number (seconds)
Default: 60

Description: If syslog-ng PE does not receive any message (acknowledgement, or other message related to the ALTP protocol) in the given timeframe, syslog-ng PE terminates the connection with the peer, and the "Connection broken" message appears in the logs of the sender (the syslog-ng PE client). This is normal, and happens when the sender does not send any new message to the receiver.

Under normal circumstances, you should not change the value of this option. The response-timeout() option can be used in source and destination drivers as well.

tls-required()
Type: yes, optional, no
Default: optional

Description: Determines whether STARTTLS is to be used during communication. If the option is set to yes, you must also configure the tls() option to specify other parameters of the TLS connection (for example, the authentication of the server and the client).

The tls-required() option can be used in source and destination drivers as well.

For example, if you configure tls-required(yes) on server side and tls-required(no) on client side, the connection is dropped. If one of them is set to optional, the configuration of the other side will decide if TLS is used or not. If both sides are set to optional, and the tls() option is properly configured, TLS encryption will be used. The following table summarizes the possible options and their results.

Note that the various parameters of the tls() option are considered in the connection only if the tls-required() settings of the peers result in TLS-encryption in the following table. In other words: the tls-required() option of ALTP determines if TLS should be used at all, while the peer-verify() option of the tls()setting determines if the TLS connection can be actually established.

tls-required() setting on the server
yes no optional
tls-required() setting on the client yes TLS-encryption rejected connection TLS-encryption
no rejected connection unencrypted connection unencrypted connection
optional TLS-encryption unencrypted connection TLS-encryption if the tls() option is set, unencrypted connection otherwise

Setting tls-required(optional) on your server allows you to receive both encrypted and unencrypted connections on the same port.

Examples for using RLTP

Caution:

Version 7.0.9 of syslog-ng Premium Edition can receive messages using Advanced Log Transfer Protocol (it is supported only as a source), sending messages is currently not implemented.

Note that in earlier versions of syslog-ng Premium Edition, Advanced Log Transfer Protocol is called Reliable Log Transfer Protocol (RLTP).

Example: Simple ALTP connection

The sender and the receiver use ALTP over the network() protocol. Since the tls() option is not configured neither on the sender nor on the receiver, the communication will be unencrypted.

Receiver configuration (syslog-ng PE server):

source s_network_altp {
        network(
            ip("127.0.0.1")
            port("5555")
            transport(altp)
            ip-protocol(4)
        );
};

Sender configuration (syslog-ng PE client):

destination d_network_rltp {
        network(
            "127.0.0.1"
            port("5555")
            transport(rltp)
            ip-protocol(4)
        );
};
Example: ALTP with TLS encryption

The following example configure a sender and a receiver to communicate using ALTP. Since the tls-required() option is set to optional on the receiver and yes on the sender, and the tls() option is configured, the communication will be TLS-encrypted. For the sender (syslog-ng PE client), reliable disk-buffering is enabled to prevent data loss.

Receiver configuration (syslog-ng PE server):

source s_syslog_altp {
        syslog(
            ip("127.0.0.1")
            port("4444")
            transport(altp(tls-required(optional)))
            ip-protocol(4)
            tls(
                peer-verify(required-trusted)
                ca-dir("/var/tmp/client/")
                key-file("/var/tmp/server/server_priv.key")
                cert-file("/var/tmp/server/server.crt")
            )
        );
};

Sender configuration (syslog-ng PE client):

destination d_syslog_rltp {
        syslog(
            "127.0.0.1"
            port("4444")
            transport(rltp(tls-required(yes)))
            ip-protocol(4)
            disk-buffer( mem-buf-size(200000) disk-buf-size(2000000) reliable(yes) )
            tls(
                peer-verify(required-trusted)
                ca-dir("/var/tmp/server/")
                key-file("/var/tmp/client/client_priv.key")
                cert-file("/var/tmp/client/client.crt")
            )
        );
};

Manipulating messages

This chapter explains the methods that you can use to customize, reformat, and modify log messages using syslog-ng Premium Edition.

Customizing message format using macros and templates

The following sections describe how to customize the names of logfiles, and also how to use templates, macros, and template functions.

Related Documents

The document was helpful.

Select Rating

I easily found the information I needed.

Select Rating