The purpose of this article is to help the investigation of syslog connection issues.
It doesn't cover the configuration of syslog sources or destinations.
Syslog connection in the view of Syslog-ng is a client-server connection to transfer standard BSD or IETF syslog formatted messages, using network() or syslog() driver.
Syslog-ng generates status and error messages of syslog connections on the internal() source.
Checking these messages in the system logs can help to find the root cause of a failed syslog connection.
The log message of a syslog connection includes the state, the IP:PORT information and the file descriptor (fd) of the connection.
The basic syslog connection messages are the followings.
Syslog connection accepted
A syslog client has connected to syslog-ng server.
Syslog connection accepted; fd='20', client='AF_INET(10.10.10.10:56928)', local='AF_INET(0.0.0.0:2000)'
A syslog connection to a remote server is established. This message appears on syslog-ng client machines.
Syslog connection established; fd='11', server='AF_INET(10.10.10.10:514)', local='AF_INET(0.0.0.0:0)'
A syslog connection is closed normally by the client or by the server. This message appears for both outgoing and incoming syslog connections.
Syslog connection closed; fd='20', client='AF_INET(10.10.10.10:56928)', local='AF_INET(0.0.0.0:2000)'
Syslog connection broken
An established syslog connection was terminated. It can be caused by various reasons, for example timeout or network issue.
Syslog connection broken; fd='15', server='AF_INET(10.10.10.10:514)', time_reopen='60'
Syslog connection failed
A syslog connection can not establish. The error message provides information about the issue.
Syslog connection failed; fd='80', server='AF_INET(192.168.1.2:6514)', error='No route to host (113)', time_reopen='60'
Usually it happens when the remote server is not accessible because of a network outage or there may be a typo in the configuration.
In such cases syslog-ng client will drop a "Syslog connection failed" message. The connection doesn't reach the server, so there is no log message can be found.
When a protocol related issue happens, it is recommended to check the logs of both the syslog server and client. In many cases the syslog server gives more exact error messages.
In such cases the connection can be established, but the server does not accept the log because of a protocol error.
The syslog-ng server may refuse the connection, because of various reasons e.g. license or max-connections() limit has been reached. In such cases, before closing the connection the server logs an error message with the reason of rejection.
Before troubleshooting a syslog connection it is recommended to double-check the followings.
The following example shows the log messages of a connection which tries to use incorrect format. The server closes the connection because the client sends BSD syslog messages instead of IETF syslog.
Syslog client messages
May 13 12:15:07 syslog-client syslog-ng: Syslog connection established; fd='18', server='AF_INET(10.10.10.10:601)', local='AF_INET(0.0.0.0:0)'
May 13 12:15:07 syslog-client syslog-ng: EOF occurred while idle; fd='18'
May 13 12:15:07 syslog-client syslog-ng: Syslog connection broken; fd='18', server='AF_INET(10.10.10.10:601)', time_reopen='60'
Syslog server messages
2019-05-13T12:15:07+02:00 syslog-server syslog-ng: Syslog connection accepted; fd='50', client='AF_INET(10.1.1.30:46746)', local='AF_INET(10.10.10.10:601)'
2019-05-13T12:15:07+02:00 syslog-server syslog-ng: Invalid frame header; header=''
2019-05-13T12:15:07+02:00 syslog-server syslog-ng: Syslog connection closed; fd='50', client='AF_INET(10.1.1.30:46746)', local='AF_INET(10.10.10.10:601)'
Syslog-ng may produce lots of internal messages on a loaded system. Syslog connection tracing can be done with the following method.
Syslog message formats: https://support.oneidentity.com/syslog-ng-premium-edition/kb/264126
Common connection issues of syslog-ng: https://support.oneidentity.com/syslog-ng-premium-edition/kb/284050
Common issues of TLS encrypted message transfer: https://support.oneidentity.com/syslog-ng-premium-edition/kb/263658