This section provides a Windows Event Collector (WEC) configuration example for using Kerberos authentication between Windows Event Forwarding (WEF) and WEC.
Example: Configuring WEC if Kerberos authentication is used
The configuration example lists the following:
-
The server and port parameters
-
Path where keytabfile is generated (/root/linux_service_account.keytab)
-
Log detail level (debug)
-
Subscription set to all computers ("*") in the Wec-Kerberos domain
-
Subscriptions-related parameters (such as connectionretry: 60.0 and batchtimeoutlimit: 900.000 seconds)
server: "testlinuxhost.testdomain.com"
port: 5985
kerberos:
keytabfile: /root/linux_service_account.keytab
log:
level: "debug"
subscriptions:
- name: "Wec-Kerberos"
computers:
- "*"
contentformat: "RenderedText"
heartbeats: 900.000
connectionretry: 60.0
batchtimeoutlimit: 900.000
queries: |
<QueryList>
<Query Id="0">
<Select Path="Application">*</Select>
</Query>
</QueryList>
For more information on the parameters and the configurable values, see Configuring Windows Event Collector.
If WEC started successfully with Kerberos authentication, the following message is displayed in the WEC debug log:
"Kerberos AUTH succeeded"
WEC clustering in syslog-ng PE
From version 7.0.23, syslog-ng Premium Edition (syslog-ng PE) supports distributing and monitoring Windows Event Forwarder connections with a TCP-level load balancer across multiple Windows Event Collector (WEC) instances (in short, WEC clustering). While using WEC clustering in syslog-ng PE, the configuration also uses a Redis key-value database to share WEC instances' state.
NOTE: Kerberos authentication does not work in a WEC cluster deployment.
Figure 5: A schematic figure of WEC clustering
NOTE: There is no direct connection between WEC and Windows instances. Instead, the Windows machines are connected to load balancers, and the load balancers are connected to WEC instances. This also means that mutual authentication is performed between Windows hosts and load balancers, and then there is also a mutual authentication performed between load balancers and WEC instances.
Limitations
|
|
Caution:
Hazard of data loss!
In syslog-ng Premium Edition (syslog-ng PE) version 7.0.23, Redis Cluster is not supported. If you attempt to set up your configuration to enable the Redis Cluster feature, your WEC cluster will not function properly. |
NOTE: The timers described the Timers on the WEC side section are not user configurable in version 7.0.23 of syslog-ng PE.
For more detailed information about WEC clustering, see the following sections:
Topics:
This section describes the principles of the working mechanism behind Windows Event Collector (WEC) clustering.
If you are new to the concept of Windows Event Collectors, see the Introduction section of this guide, or the sections following the introduction for further details.
How does WEC clustering work?
Windows Event Forwarders are connected to load balancers, and the load balancers are connected to Windows Event Collector (WEC) instances. When a WEC instance is disconnected from a load balancer, the load balancer detects the disconnected WEC instance, and forwards events to a different WEC instance. The new WEC instance can continue event requests from the last position saved by the disconnected WEC. To solve the redirection problem, you must have WEC instances that can be configured to share their states. Because it is possible for WEC instances to store their state in a Redis key-value database, syslog-ng Premium Edition (syslog-ng PE) utilizes this Redis feature and shares WEC instances' state across such a Redis key-value database when performing WEC clustering.
In Windows Event Forwarders, the subscription's status at a given time is recorded in a file (or a Redis key-value database, depending on your use case) that is commonly called a state. Accordingly, while discussing Windows Event Collector (WEC) clustering , these sections mention two types of states: file-based states and Redis states.
NOTE: File-based states are local states and are not synchronized between multiple WEC instances (as opposed to shared states) File-based states can only be used locally, by a single WEC instance at a time.
The three data types within file-based states
States that are recorded in a file (that is, file-based states) consist of three data types:
-
An event source (also called a host), where the events are received.
-
A subscription ID.
-
A bookmark (also called a position).
In case of file-based states, this information is stored within files (in the stateDir directory). Each file stored in the stateDir directory contains subscription IDs and positions to the given host, one file each.
Example: data structure within the stateDir directory
stateDir\win1: subscription_1 - position_1 subscription_2 - position_2 ... subscription_n - position_n
NOTE: Real state files store these data in a different format. This format is only true for the information structure within the stateDir directory.
Mapping the file-based state structure to Redis states
If the user wants to use Redis states instead of file-based states while WEC clustering, the information has to be mapped to HSET data structures, a data structure native to Redis. Event sources are capable of identifying HSET data structures in file-based states' data structure contents, and the same state structure of subscription IDs and positions to the given host is mapped to Redis states as follows:
-
In Redis, the Redis connections are mapped to the stateDir directory of the file-based state.
-
In Redis, the HSETs are mapped to the file name of the event source (or host) of the file-based state.
-
In Redis, the HKEYs within the HSETs are mapped to subscription IDs of the file-based state.
-
In Redis, the HVALUEs within the HSETs are mapped to bookmarks (positions) of the file-based state.
What happens if the WEC instance is disconnected from a load balancer?
When a WEC instance is disconnected from a load balancer, the subscriptions connected to this instance are redirected to a different available WEC instance. When the new WEC instance receives the subscription, the WEC instance looks the subscription up in the Redis state, then populates the position from Redis. After this, the WEC instance continues processing requests from the last position loaded from the Redis backend. For each event (batch), the WEC instance writes bookmarks to Redis.
NOTE: Redis manages concurrent read or write operations.
What happens if Redis becomes unavailable for a WEC?
If for some reason, Redis becomes unavailable for a WEC istance, the WEC instance performs the following, in this order:
-
The WEC instance stops the listener.
-
The WEC instance stops all TCP connections established between the load balancer and the WEC instance.
-
The WEC instance stops sending messages to syslog-ng Premium Edition (syslog-ng PE).
-
The WEC instance starts to ping Redis.
NOTE: The load balancer is connected to the listener, so when the WEC instance is disconnected, the load balancer redirects the Windows event source to a different WEC instance. When Redis becomes available again, the load balancer detects it, and starts forwarding the events to the original WEC instance.
