The following destination drivers can use the disk-buffer option: elasticsearch2(), file(), hdfs(), http(), kafka(), mongodb(), program(), riemann(), sentinel(), smtp(),sql(), unix-dgram(), and unix-stream(). The network(), syslog(), tcp(), and tcp6() destination drivers can also use the disk-buffer option, except when using the udp transport method. (The other destinations or protocols do not provide the necessary feedback mechanisms required for the disk-buffer option.)
To enable using the reliable disk-buffer option, use the disk-buffer(reliable(yes)) parameter in the destination. Use the reliable disk-buffer option if you do not want to lose logs in case of reload/restart, an unreachable destination or syslog-ng PE crash. This solution provides a slower, but reliable disk-buffer option. It is created and initialized at startup and gradually grows as new messages arrive. The filename of the reliable disk-buffer file is the following: <syslog-ng path>/var/syslog-ng-00000.rqf.
Example: Example for using the reliable disk-buffer option
destination d_BSD {
network(
"127.0.0.1"
port(3333)
disk-buffer(
mem-buf-size(10000)
disk-buf-size(2000000)
reliable(yes)
)
);
};
For more details on the differences between the normal and the reliable disk-buffer options, see About disk queue files.
The following destination drivers can use the disk-buffer option: elasticsearch2(), file(), hdfs(), http(), kafka(), mongodb(), program(), riemann(), sentinel(), smtp(),sql(), unix-dgram(), and unix-stream(). The network(), syslog(), tcp(), and tcp6() destination drivers can also use the disk-buffer option, except when using the udp transport method. (The other destinations or protocols do not provide the necessary feedback mechanisms required for the disk-buffer option.)
If the reliable() option is not set, by default a normal disk-buffer is created. To explicitly enable the normal disk-buffer option, use the disk-buffer(reliable(no)) parameter in the destination. Use the normal disk-buffer option if you want a solution that is faster than the reliable disk-buffer option. In this case, the process will be less reliable and it is possible to lose logs in case of syslog-ng PE crash. The filename of the normal disk-buffer file is the following: <syslog-ng path>/var/syslog-ng-00000.qf.
Example: Example for using the normal disk-buffer option
When using the plugin for the disk-buffer file
destination d_BSD {
network(
"127.0.0.1"
port(3333)
disk-buffer(
mem-buf-length(10000)
disk-buf-size(2000000)
reliable(no)
)
);
};
For more details on the differences between the normal and the reliable disk-buffer options, see About disk queue files.
This section describes how to get information about disk-buffer files used in syslog-ng Premium Edition (syslog-ng PE).
NOTE: While reading this section, consider that the default installation path used in the commands and syslog-ng PE files is /opt/syslog-ng.
Topics:
This section describes information about disk-buffer files used in syslog-ng Premium Edition (syslog-ng PE).
The following list contains information about how disk-buffer files are used in syslog-ng PE:
-
You can configure disk-buffer() for a remote destination in the destination() statement.
For more information about an example of configuring disk-buffer() for a remote destination in the destination() statement, see disk-buffer().
- By default, syslog-ng PE creates disk-buffer files under /opt/syslog-ng/var directory, unless dir() option is set in disk-buffer().
- The filenames are generated automatically by syslog-ng PE with the extensions .qf for a normal disk-buffer and .rqf for a reliable disk-buffer.
- The disk-buffer file stores processed log messages in the format in which they would have been sent out to the destination, but doesn't store information about the destination.
