-
Title
How to configure syslog-ng to send the data to Apache Kafka. -
Description
How to configure syslog-ng to send the data to Apache Kafka. -
Resolution
We highly recommend our Professional Services Organization for application configuration or implementation advice. For information on how to contact them please visit https://support.oneidentity.com//professional-services-product-select
Here is some information from the Administration guide to get you started and a link to the information:
To publish messages from syslog-ng PE to Apache Kafka1. If you want to use the Java-based modules of syslog-ng PE (for example, the Elasticsearch, HDFS, or Kafka destinations), download and install the Java Runtime Environment (JRE), 1.7 (or newer).
The Java-based modules of syslog-ng PE are tested and supported when using the Oracle implementation of Java. Other implementations are untested and unsupported, they may or may not work as expected.2. Download the latest stable binary release of the Apache Kafka libraries (version 0.9 or newer) from http://kafka.apache.org/downloads.html.
3. Extract the Apache Kafka libraries into a single directory. If needed, collect the various .jar files into a single directory (for example, /opt/kafka/lib/) where syslog-ng PE can access them. You must specify this directory in the syslog-ng PE configuration file.
4. Check if the following files in the Kafka libraries have the same version number: slf4j-api-.jar, slf4j-log4j12-.jar. If the version number of these files is different, complete the following steps:
a. Delete one of the files (for example, slf4j-log4j12-.jar).b. Download a version that matches the version number of the other file (for example, 1.7.6) from the official SLF4J distribution.c. Copy the downloaded file into the directory of your Kafka library files (for example, /opt/kafka/lib/).Here is the direct link to the section in the Administration Guide: kafka: Publishing messages to Apache Kafka