syslog-ng Premium Edition 7.0.14 - Windows Event Collector Administration Guide

Flow control

The Windows Event Collector tool applies flow control to minimize event log loss.

WEC regularly (in every second) polls the datagram socket that will receive the Windows events to check whether it exists already. Once the socket has been created (syslog-ng PE has started up), WEC connects to the socket and accepts the incoming connections from the Windows hosts. If the datagram socket does not exist, WEC refuses the incoming connections.

If the socket exists (syslog-ng PE is running) but syslog-ng PE does not read the Unix datagram socket, WEC fills up the kernel buffer of the datagram socket and stores queuesize amounts of log messages in the memory. When all buffers are full, WEC stops reading messages from the HTTP connections to prevent message loss.

The buffer size of a datagram socket is determined by certain Linux kernel parameters: the value of rmem_* (max/default) and the count of net.unix.max_dgram_qlen.

Reliability

WEC flags a message as delivered once it has put the message in the socket buffer. If syslog-ng PE does not read the socket for some reason (for example, because it is still flow-controlled) and syslog-ng PE is stopped, the contents of this socket (that is, the messages that are in the kernel buffer, unread by syslog-ng PE) will be lost.

This is why in cases when a restart is necessary, it is recommended to stop the Windows Event Collector and syslog-ng PE in the following order:

  1. Windows Event Collector

  2. syslog-ng PE

While it is not guaranteed that syslog-ng PE has read all sockets by the time you stop it, at least you can maximize the chances that it has.

Performance

Performance is dependent on the number of event log messages that the Windows hosts send to WEC and the capabilities of the XML parser.

Our performance measurements indicate that syslog-ng PE's XML parser is capable of parsing 15,000-20,000 events/second. The exact capacity of the XML parser depends on the complexity of the Windows log messages, as well as the performance of the hardware that syslog-ng PE and WEC are running on. When the limit of 15,000-20,000 events/seconds is reached, a workaround is recommended.

As the value set in the batchsizelimit parameter is treated only as a recommendation by the Windows hosts, there is no direct way to control the amount of messages arriving from the event source computers.

A possible workaround is to launch multiple WEC servers and create multiple windowsevent() sources in syslog-ng PE. That way, you can distribute your Windows hosts across multiple WEC and syslog-ng PE servers, decreasing the load on individual servers.

To run multiple WEC services per syslog-ng PE service, you need to create your own init script. This is because the init script that comes with WEC enables you to run only a single WEC service per syslog-ng PE service.

Limitations

The Windows Event Collector for syslog-ng PE currently has the following limitations:

  • Only source-initiated push subscriptions are supported (Windows hosts connect to the WEC server).

    An advantage of this, however, is that this requires less firewall rules.

  • Only HTTPS and SSL certificate based authentication are supported. Kerberos authentication is not supported at the moment.

  • The compression of events is not supported.

  • The batchsizelimit and batchtimeoutlimit options are not enforced on the Windows host side: Windows is handling these values only as a recommendation.

  • On Windows 7 and Windows 2008 platforms, there is a known issue. After several reconnects (if WEC is restarted quickly), the remote sender can stop forwarding the logs for a certain period of time. In this case, restarting the Windows RM service can help.

    This issue can also occur between two Windows machines. It has been reported to Microsoft and is awaiting resolution.

Troubleshoot Windows Event Collector

When you experience issues while using WEC, run WEC in debug mode to get detailed log messages.

  1. Set the log level to debug:

    log:
      level: "debug"
  2. Start WEC.

    At every refresh interval, the following debug messages should be displayed:

    DEBUG   subscriptionEndpoint    {"clientAddress": "..."}
    DEBUG   actionHandler   {"messageID": "...", "action": "http://schemas.xmlsoap.org/ws/2004/09/enumeration/Enumerate"}
    DEBUG   enumerate

    This means that the client has connected and requested the subscription list.

  3. If you cannot see these messages within the refresh interval, you should check the following channels in the client's event viewer:

    • Applications and Services Logs\Microsoft\Windows\Eventlog-ForwardingPlugin

    • Applications and Services Logs\Microsoft\Windows\Windows Remote Management

Some common error codes and their explanations:

  • 5004: A channel specified in the query XML does not exist or cannot be read on the Windows client. This can be caused by the "Network Service" not having permission to read the security log.

    Add the "Network Service" account to the Event Log Readers group, and restart the computer for changes to take effect.

  • 15008: The query XML of the subscription is invalid.

  • 995 (HTTP error 12186): The "Network Service" does not have permission to read the client certificate.

  • HTTP error 403: If everything is set correctly, then it might be possible that a proxy is set and the forwarder tries to connect to the proxy instead of WEC.

    TIP:

    Sometimes proxy settings are not displayed in any GUI window. Check them using netsh winhttp show proxy. To reset proxy settings, use netsh winhttp reset proxy.

Related Documents