syslog-ng Open Source Edition 3.30 - Administration Guide

stomp() destination options

The stomp() driver publishes messages using the Simple (or Streaming) Text Oriented Message Protocol (STOMP).

The stomp() destination has the following options:

ack()

Description: Request the STOMP server to acknowledge the receipt of the messages. If you enable this option, then after sending a message, syslog-ng OSE waits until the server confirms that it has received the message. This delay can seriously limit the performance of syslog-ng OSE if the message rate is high, and the server cannot acknowledge the messages fast enough.

body()

Description: The body of the STOMP message. You can also use macros and templates.

destination()

Description: The name of the destination (message queue) on the STOMP server. It can include macros and templates.

disk-buffer()

Description: This option enables putting outgoing messages into the disk buffer of the destination to avoid message loss in case of a system failure on the destination side. It has the following options:

destination d_demo {
    network(
        "127.0.0.1"
        port(3333)
        disk-buffer(
            mem-buf-size(10000)
            disk-buf-size(2000000)
            reliable(yes)
            dir("/tmp/disk-buffer")
        )
    );
};

destination d_demo {
    network(
        "127.0.0.1"
        port(3333)
           disk-buffer(
            mem-buf-length(10000)
            disk-buf-size(2000000)
            reliable(no)
            dir("/tmp/disk-buffer")
        )
    );
};

batch-lines()

Description: Specifies how many lines are flushed to a destination in one batch. The syslog-ng OSE application waits for this number of lines to accumulate and sends them off in a single batch. Increasing this number increases throughput as more messages are sent in a single batch, but also increases message latency.

For example, if you set batch-lines() to 100, syslog-ng OSE waits for 100 messages.

If the batch-timeout() option is disabled, the syslog-ng OSE application flushes the messages if it has sent batch-lines() number of messages, or the queue became empty. If you stop or reload syslog-ng OSE or in case of network sources, the connection with the client is closed, syslog-ng OSE automatically sends the unsent messages to the destination.

Note that if the batch-timeout() option is enabled and the queue becomes empty, syslog-ng OSE flushes the messages only if batch-timeout() expires, or the batch reaches the limit set in batch-lines().

For optimal performance, make sure that the syslog-ng OSE source that feeds messages to this destination is configured properly: the value of the log-iw-size() option of the source must be higher than the batch-lines()*workers() of the destination. Otherwise, the size of the batches cannot reach the batch-lines() limit.

batch-timeout()

Description: Specifies the time syslog-ng OSE waits for lines to accumulate in the output buffer. The syslog-ng OSE application sends batches to the destinations evenly. The timer starts when the first message arrives to the buffer, so if only few messages arrive, syslog-ng OSE sends messages to the destination at most once every batch-timeout() milliseconds.

hook-commands()

Description: This option makes it possible to execute external programs when the relevant driver is initialized or torn down. The hook-commands() can be used with all source and destination drivers with the exception of the usertty() and internal() drivers.

Using the hook-commands() when syslog-ng OSE starts or stops

To execute an external program when syslog-ng OSE starts or stops, use the following options:

Using the hook-commands() when syslog-ng OSE reloads

To execute an external program when the syslog-ng OSE configuration is initiated or torn down, for example, on startup/shutdown or during a syslog-ng OSE reload, use the following options:

source {
   network(transport(udp)
	hook-commands(
          startup("iptables -I LOGCHAIN 1 -p udp --dport 514 -j ACCEPT")
          shutdown("iptables -D LOGCHAIN 1")
        )
     );
};

host()

Description: The hostname or IP address of the STOMP server.

password()

Description: The password used to authenticate on the STOMP server.

persistent()

Description: If this option is enabled, the STOMP server or broker will store the messages on its hard disk. That way, the messages will be retained if the STOMP server is restarted, if the message queue is set to be durable on the STOMP server.

port()

Description: The port number of the STOMP server.

retries()

Description: The number of times syslog-ng OSE attempts to send a message to this destination. If syslog-ng OSE could not send a message, it will try again until the number of attempts reaches retries, then drops the message.

throttle()

Description: Sets the maximum number of messages sent to the destination per second. Use this output-rate-limiting functionality only when using disk-buffer as well to avoid the risk of losing messages. Specifying 0 or a lower value sets the output limit to unlimited.

username()

Description: The username used to authenticate on the STOMP server.

value-pairs()

scope("selected-macros" "nv-pairs")

Description: The value-pairs() option creates structured name-value pairs from the data and metadata of the log message. For details on using value-pairs(), see Structuring macros, metadata, and other value-pairs.

Sumo Logic destinations: sumologic-http() and sumologic-syslog()

From version 3.27.1, the syslog-ng Open Source Edition (syslog-ng OSE) application can send log messages to Sumo Logic, a cloud-based log management and security analytics service, by using the sumologic-http() and sumologic-syslog() destinations.

destination d_sumo_http {
  sumologic-http(
    collector("ZaVnC4dhaV3_[...]UF2D8DRSnHiGKoq9Onvz-XT7RJG2FA6RuyE5z4A==")
    deployment("eu")
    tls(peer-verify(yes) ca-dir('/etc/syslog-ng/ca.d'))
  );
};

destination d_sumo_syslog {
  sumologic-syslog(
    token("rqf/bdxYVaBLFMoU39[...]CCC5jwETm@41123")
    deployment("eu")
    tls(peer-verify(yes) ca-dir('/etc/syslog-ng/ca.d'))
  );
};

@include "scl.conf"

sumologic-http()

The sumologic-http() and sumologic-syslog() destinations send log messages to Sumo Logic, a cloud-based log management and security analytics service.

Using the sumologic-http() destination, you can send data to the Sumo Logic service by utilizing a Hosted Collector hosted by Sumo Logic.

For more information about the sumologic-http() destination, see sumologic-syslog().

log {
  source { system(); };

  destination {
    sumologic-http(
      collector("UNIQUE-HTTP-COLLECTOR-CODE-AS-PROVIDED-BY-sumologic")
      deployment("ENDPOINT")
      tls(peer-verify(yes) ca-dir('/etc/syslog-ng/ca.d'))
    );
  };
};

sumologic-syslog()

The sumologic-http() and sumologic-syslog() destinations send log messages to Sumo Logic, a cloud-based log management and security analytics service.

Using the sumologic-syslog() destination, you can send data (both in JSON and in non-JSON format) to the Sumo Logic service.

For more information about the sumologic-http() destination, see sumologic-http().

log {
  source { system(); };

  destination{
    sumologic-syslog(token("USER-TOKEN-AS-PROVIDED-BY-sumologic")
      deployment("ENDPOINT")
      tls(peer-verify(required-trusted) ca-dir('/etc/syslog-ng/ca.d'))
    );
};
};

log {
  source{ system(); };

  destination{
    sumologic-syslog(token("USER-TOKEN-AS-PROVIDED-BY-sumologic")
      deployment("ENDPOINT")
      tls(peer-verify(required-trusted) ca-dir('/etc/syslog-ng/ca.d'))
      template("$(format-json --scope all-nv-pairs)")
    );
  };
};