Printing the content of disk-buffer files
This section describes printing the content of the disk-buffer files used in syslog-ng Premium Edition(syslog-ng PE).
Command syntax
The command syntax for printing the content of the disk-buffer files used in syslog-ng PE looks like this:
/opt/syslog-ng/bin/dqtool cat DISK_QUEUE_FILE
Short example output for printed content
Example: short output that shows the printed content of the disk-buffer files used in syslog-ng PE
The following short output example shows the printed content of the disk-buffer files used in syslog-ng PE:
/opt/syslog-ng/bin/dqtool cat /opt/syslog-ng/var/syslog-ng-00000.rqf Reliable disk-buffer state loaded; filename='/opt/syslog-ng/var/syslog-ng-00000.rqf', queue_length='2952', size='-437712' Jul 31 12:33:48.226 10.21.10.10 <382019-07-31T12:33:36 localhost prg00000[1234]: seq: 0000000838, thread: 0000, runid: 1564569216, stamp: 2019-07-31T12:33:36 PADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADDPADD ...
Orphan disk-buffer files
This section describes orphan disk-buffer files used in syslog-ng Premium Edition(syslog-ng PE).
In certain situations (for example, after modifying the disk-buffer configuration or losing the persist information), syslog-ng PE creates a new disk-buffer file instead of using the old one.
To discover these new disk-buffer files (also called orphan disk-buffer files), get the list of disk-buffer files from the persist file, and compare it with the content of the disk-buffer files' saving directory.
For more information about getting the list of disk-buffer files from the persist file, see Getting the list of disk-buffer files.
Discovering the new disk-buffer files (orphan disk-buffer files)
The following examples show the difference between the list of disk-buffer files from the persist file and the content of the disk-buffer files' saving directory:
Example: difference between the list of disk-buffer files from the persist file and the content of the disk-buffer files' saving directory
Disk-buffer file list from persist file:
afsocket_dd_qfile(stream,10.21.10.112:514) = { "queue_file": "/opt/syslog-ng/var/syslog-ng-00001.rqf" }
Disk-buffer files' saving directory content:
# ls -l /opt/syslog-ng/var/*qf
-rw------- 1 root root 2986780 Jul 31 12:30 /opt/syslog-ng/var/syslog-ng-00000.qf
-rw------- 1 root root 2000080 Jul 31 12:31 /opt/syslog-ng/var/syslog-ng-00000.rqf
-rw------- 1 root root 4096 Aug 1 11:09 /opt/syslog-ng/var/syslog-ng-00001.rqfThe disk-buffer files syslog-ng-00000.qf and syslog-ng-00000.rqf don't exist in the persist file. These two files are the newly created disk-buffer files that we also call orphan disk-buffer files.
How to empty disk-buffer files
This section describes how to empty disk-buffer files used in syslog-ng Premium Edition (syslog-ng PE).
|
|
Caution:
Hazard of data loss! You must stop log reception to be able to empty a disk-buffer. If you fail to stop log reception before emptying a disk-buffer, your newly received log messages may get stored in the disk-buffer, overwriting your previous log messages. To avoid log loss, One Identity recommends that you redirect your logs to a different syslog server when emptying your disk-buffer files. |
NOTE: Consider the following while reading this section:
This section uses a simple example configuration with one source and one destination with disk-buffer.
If you are not aware of disk-buffers or you're not sure which of your destinations use disk-buffer, One Identity recommends that you do not proceed with the procedure of emptying your disk-buffer files. Instead, One Identity recommends that you contact our Support Team and open a service request. When opening the service request, describe your issue and attach a collected debug bundle from your system.
For more information about collecting a debug bundle for Microsoft Windows, see How to create a syslog-ng debug bundle archive on Windows operating system.
For more information about collecting a debug bundle for Linux or Unix OS, see How to create a syslog-ng debug bundle on Linux Or Unix operating system.
Recommendation
One Identity recommends that you empty your disk-buffer files before you begin the following:
-
Upgrading syslog-ng Premium Edition (syslog-ng PE) from version 6 to 7.
-
Changing the configuration of a remote destination with disk-buffer.
-
Applying a solution that includes the removal of the syslog-ng PE persistent file.
Example configuration for emptying disk-buffer files
The syslog-ng PE application uses the following example configuration to describe how to empty disk-buffer files:
source s_net {
network();
};
destination d_logserver {
network("10.21.10.20" port(514) disk-buffer( disk-buf-size(2000000) ) );
};
log {
source(s_net);
destination(d_logserver);
};To empty disk-buffer files,
-
Name the disk-buffer file to empty and the destination statement using it.
If you are not sure about which disk-buffer file to empty, or the destination statement using the disk-buffer file in question, you can use one of the following methods:
-
Check the list and the status of the disk-buffer files.
Examples
-
Non-empty disk-buffer file
Disk-buffer state loaded; filename='/opt/syslog-ng/var/syslog-ng-00000.qf', qout_length='0', qbacklog_length='0', qoverflow_length='0', qdisk_length='3006'
-
IP:PORT information of the destination with the disk-buffer in use
afsocket_dd_qfile(stream,10.21.10.20:514) = { "queue_file": "/opt/syslog-ng/var/syslog-ng-00000.qf" }
For more information about getting information about disk-buffer files, see Useful information about disk-buffers.
-
-
Find the destination statement in the syslog-ng PE configuration using the IP:PORT information.
destination d_logserver { network("10.21.10.20" port(514) disk-buffer( disk-buf-size(2000000) ) ); };
-
-
Locate the log statements that use the destination statement you named previously.
-
Disable the sources in the log statements.
Add '#' at the beginning of all source() entries in the log paths.
log { #source(s_net); destination(d_logserver); } -
Reload syslog-ng PE by entering the /opt/syslog-ng/sbin/syslog-ng-ctl reload command.
-
Check the disk-buffer file status.
For more information, see Getting status information of disk-buffer files.
-
To enable the sources again, remove '#' from the log paths and reload syslog-ng PE.
Enabling memory buffering
To enable memory buffering, use the log-fifo-size() parameter in the destination. All destination drivers can use memory buffering. Use memory buffering if you want to send logs to destinations where the disk-buffer option is not available, if you want the fastest solution, and if syslog-ng PE crash or network downtime is never expected. In these cases, losing logs is possible. This solution does not use the disk-buffer option. Instead, logs are stored only in the memory.
Example: Example for using memory buffering
destination d_BSD {
network(
"127.0.0.1"
port(3333)
log-fifo-size(10000)
);
};