-
Title
Troubleshooting Syslog-ng -
Description
Troubleshooting Syslog-ng
-
Resolution
Basic troubleshooting
Syslog-ng has an advanced internal logging mechanism. In case of event info, warning, error, fatal or debug messages are generated. All messages generated internally by syslog-ng use a special source called internal(). Syslog-ng is shipped with a configuration which collects the internal logs into /var/log/messages.
If you're unsure, find the source uses internal() driver. Search for log statement uses the "internal" source and check the destination.
Start the investigation by checking for issues in internal logs, /var/log/messages.
egrep -i 'syslog-ng.*(error|fatal)' /var/log/messages
Verbose logging
If you are trying to solve configuration problems, the verbose messages are usually sufficient.
You can turn on verbose logging on a running syslog-ng instance with the following command.
/opt/syslog-ng/sbin/syslog-ng-ctl verbose --set=on
When you have finished, you can turn off verbose logs with:
/opt/syslog-ng/sbin/syslog-ng-ctl verbose --set=off
Debug logging
In certain situations you may need to analyze debug messages, for example to troubleshoot connection issues or to verify message processing.
Note: PE 6 - full message processing, PE 7 only debug logs of incoming and outgoing messagesYou can turn on debug logging on a running syslog-ng instance with the following command.
/opt/syslog-ng/sbin/syslog-ng-ctl debug --set=on
When you have finished turn off debug mode with:
/opt/syslog-ng/sbin/syslog-ng-ctl debug --set=off
Debug logging can produce a huge amount of logs. It is strongly recommended to turn it on only for short troubleshooting sessions.
Trace logging for syslog-ng PE 7
You can turn on trace logging (message processing) on a running syslog-ng PE 7 instance with the following command.
Note: You may want to enable debug logging as well to see debug logs of incoming and outgoing messages./opt/syslog-ng/sbin/syslog-ng-ctl trace --set=on
When you have finished turn off trace mode with:
/opt/syslog-ng/sbin/syslog-ng-ctl trace --set=off
Trace logging produces a huge amount of logs. It is strongly recommended to turn it on for short troubleshooting sessions.
Starting syslog-ng in debug mode
When your syslog-ng instance fails to start for some reason, you can start it up in debug mode. It is also useful to troubleshoot environmental issues, for example in case of a java destination.
Note, that if you are running syslog-ng with a non-root user, you have to run the following commands with the appropriate user.
If that user does not have permission to open a terminal, go to next section.
Make sure that no syslog-ng instance is running and execute:
/opt/syslog-ng/sbin/syslog-ng -Fedv
Press CTRL + C to terminate syslog-ng, if it was not terminated by itself.
Redirecting the output to a file
Debug mode generates huge amount of log messages. It's recommended to redirect the output to a log file.
/opt/syslog-ng/sbin/syslog-ng -Fedv 2>&1 | tee -a syslog-dbg.log
Starting syslog-ng in debug mode with non-root user without shell
Depending on your system, one of the following methods should work.
With sudo
sudo -u USERNAME /opt/syslog-ng/sbin/syslog-ng -Fedv
With su
su - USERNAME -c "/opt/syslog-ng/sbin/syslog-ng -Fedv"
With systemd
Modify systemd service file located at /usr/lib/systemd/system/syslog-ng.service or /lib/systemd/system/syslog-ng.service, depending on your Linux distribution.
- Find ExecStart= parameter in [Service] section and add debug options '-d -v'.
ExecStart=/opt/syslog-ng/sbin/syslog-ng -F --no-caps --enable-core $SYSLOGNG_OPTIONS -d -v
- Reload systemd configuration
systemctl daemon-reload
- Restart syslog-ng
systemctl restart syslog-ng
- Check logs
journalctl -u syslog-ng