Caution:

This destination is deprecated and will be removed from a future version of syslog-ng PE. We recommend using the elasticsearch-http: Sending messages to Elasticsearch HTTP Event Collector destination instead.

Starting with version 5.63.7 of syslog-ng PE can directly send log messages to Elasticsearch, allowing you to search and analyze your data in real time, and visualize it with Kibana.

NOTE: Typically, only the central syslog-ng PE server uses this destination. For more information on the server mode, see Server mode.

Note the following limitations when using the syslog-ng PE elasticsearch2 destination:

  • This destination is only supported on the Linux platform.

    This destination is only supported on the Linux platforms that use the linux glibc2.11 installer, including: Red Hat ES 7, Ubuntu 14.04 (Trusty Tahr).

  • Since syslog-ng PE uses Java libraries, the elasticsearch2 destination has significant memory usage.

  • The log messages of the underlying client libraries are available in the internal() source of syslog-ng PE.

Declaration
@module mod-java
@include "scl.conf"

elasticsearch2(
    index("syslog-ng_${YEAR}.${MONTH}.${DAY}")
    type("test")
    cluster("syslog-ng")
);
Example: Sending log data to Elasticsearch version 2.x and above

The following example defines an elasticsearch2 destination that sends messages in transport mode to an Elasticsearch server running on the localhost, using only the required parameters.

@module mod-java
@include "scl.conf"

destination d_elastic {
    elasticsearch2(
        index("syslog-ng_${YEAR}.${MONTH}.${DAY}")
        type("test")
    );
};

The following example sends 10000 messages in a batch, in transport mode, and includes a custom unique ID for each message.

@module mod-java
@include "scl.conf"

options {
    threaded(yes);
    use-uniqid(yes);
};

source s_syslog {
    syslog();
};

destination d_elastic {
    elasticsearch2(
        index("syslog-ng_${YEAR}.${MONTH}.${DAY}")
        type("test")
        cluster("syslog-ng")
        client-mode("transport")
        custom-id("${UNIQID}")
        flush-limit("10000")
    );
};

log {
    source(s_syslog);
    destination(d_elastic);
    flags(flow-control);
};
Example: Sending log data to Elasticsearch using the HTTP REST API

The following example send messages to Elasticsearch over HTTP using its REST API:

@include "scl.conf"

source s_network {
    network(port(5555));
};

destination d_elastic {
    elasticsearch2(
        client-mode("http")
        cluster("es-syslog-ng")
        index("x201")
        cluster-url("http://192.168.33.10:9200")
        type("slng_test_type")
        flush-limit("0")
    );
};

log {
    source(s_network);
    destination(d_elastic);
    flags(flow-control);
};

The elasticsearch2() driver is actually a reusable configuration snippet configured to receive log messages using the Java language-binding of syslog-ng PE. For details on using or writing such configuration snippets, see Reusing configuration blocks. You can find the source of the elasticsearch configuration snippet on GitHub. For details on extending syslog-ng PE in Java, see the Getting started with syslog-ng development guide.

NOTE: If you delete all Java destinations from your configuration and reload syslog-ng, the JVM is not used anymore, but it is still running. If you want to stop JVM, stop syslog-ng and then start syslog-ng again.