This section describes the process of converting file-based states to Redis states, and the related potential limitations and issues.
For more information about file-based states and Redis states, see States in WEC clustering (file-based states and Redis states).
Conversion steps and limitations
During the conversion process, you move your existing file-based states into Redis states. The WEC binary supports moving the file-based states with a dedicated command line option (-i):
root@ubuntu1:/opt/syslog-ng/var/wecstate# /opt/syslog-ng/libexec/wec -h Usage of /opt/syslog-ng/libexec/wec:
-c string
Configuration file name (default "/opt/syslog-ng/etc/wec.yaml")
-i Initializing shared state from stateDir
-s string
Persistent state directory (default "/opt/syslog-ng/var/wecstate")
-v WEC version
To convert file-based states to Redis states
-
Load your existing file-based states from your stateDir directory.
-
Connect to Redis server.
NOTE: If your Redis server is not available, the WEC will not start at all, and the conversion process stops.
-
Save your previously loaded file-based states into a Redis HSET.
-
Move the files containing your WEC states to your stateDir/.converted folder.
|
|
CAUTION: If moving the WEC states to your stateDir/.converted folder is unsuccessful, your file-based state will contain out-of-date data. If the final conversion step is unsuccessful, and you have your -i option enabled, the out-of-date data in your file-based states overwrite the data in your Redis states after your next WEC restart (which results in message duplication). To avoid this, remove the -i command line option before you restart your WEC following your state conversion. |
NOTE: If for some reason, your WEC cluster configuration does not work as expected, your previous file-base states will not be available if you delete them. Instead, One Identity recommends moving the files to the folder, where they will be available for recovery if needed.
NOTE: State conversion has two possible results:
If any of the state conversion steps is unsuccessful, the WEC instance stops with an error. As a result, even if the file-based state is successfully saved to Redis, but moving the files is unsuccessful, the WEC instance stops and prints an error message:
Failed to move state files, remove it manually
This is an expected behavior, developed to avoid unwanted, huge message duplication and related issues. Instead of message duplication, if the file-based state is converted, but moving the state files is unsuccessful for some reason, you can re-initialize Redis with the state files that contain old bookmarks.
This section describes an example use case for Windows Event Collector (WEC) clustering .
For more detailed information about the working mechanism of WEC clustering, see The working mechanism of Windows Event Collector (WEC) clustering.
Limitations
|
|
Caution:
Hazard of data loss!
In syslog-ng Premium Edition (syslog-ng PE) version 7.0.23, Redis Cluster is not supported. If you attempt to set up your configuration to enable the Redis Cluster feature, your WEC cluster will not function properly. |
Configuration
In the example use case for WEC clustering, the following configuration is used:
bookmarks:
backend: redis
uri: 192.168.0.14:6379
password: pwd #optional
NOTE: In the example use case, multiple WEC instances are running behind the load balancer. Load balancing in this configuration will only work with multiple WEC instances if you have the same Redis backend configured on all WEC instances, with the same subscription used by all of the WEC instances. Otherwise, you will encounter message duplication.
Required components for a functional WEC cluster configuration
For a functional WEC cluster configuration, the required components are the following:
-
WEC instances, with the following considerations:
-
The WEC instances must have a shared state (on the Redis backend).
-
The WEC instances must be configured to use the same subscription.
The subscriptions part of the configuration should be the same in every WEC instance, for example:
subscriptions:
- batchsizelimit: 1
batchtimeoutlimit: 1.0
computers:
- '*'
connectionretry: 1.0
contentformat: RenderedText
heartbeats: 3.0
name: wec-1-subscription
queries: "<QueryList>\n <Query Id=\"0\">\n <Select Path=\"Application\">*</Select>\n\
\ </Query>\n</QueryList>\n"readexistingevents: false
* wec is connected to LoadBalancer (server and port is set to Load Balancer)
-
Windows instances that forward requests to load balancers.
-
An appropriately installed, set up, maintained, and monitored TCP level load balancer.
The customer has the following responsibilities:
-
Installing the TCP level load balancer.
-
Setting up the TCP level load balancer.
-
Maintaining the TCP level load balancer.
-
Monitoring the TCP level load balancer.
-
Configuring the SSL certificates appropriately.
NOTE: When configuring your SSL certificates while WEC clustering, consider that the load balancer functions as a proxy in your configuration.
For more information about configuring SSL certificates for Windows Event Collectors, see Generating SSL certificates for Windows Event Collector.
-
An appropriately set up, configured, maintained, and monitored Redis server.
The customer has the following responsibilities:
-
Setting up the Redis server.
-
Configuring the Redis server.
-
Maintaining the Redis server.
-
Monitoring the Redis server.
This section provides troubleshooting information and solutions in connection with Windows Event Collector (WEC) clustering .
This section describes the possible log messages you may get while using Windows Event Collector (WEC) clustering with syslog-ng Premium Edition (syslog-ng PE), and why the WEC sends them.
-
If Redis is not available during startup, the WEC instance cannot start. In this case, you will get a similar log message:
2020-11-16T21:24:03.843Z FATAL state/redisstate.go:17 RedisConn: Error connecting to Redis {"error": "RedisConn: connection failure: dial tcp 192.168.0.14:6379: connect: connection refused"}
-
If Redis is disconnected, you will get a similar log message:
2020-11-16T21:11:12.818Z ERROR state/redisconn.go:55 RedisConn: dial failed {"error": "dial tcp 192.168.0.14:6379: connect: connection refused"}
-
If you are trying to ping Redis periodically (in this case, the ping period is 1 second), you will get a similar log message:
2020-11-16T21:11:12.818Z DEBUG state/redisconn.go:115 RedisConn is still disconnected
2020-11-16T21:11:13.819Z ERROR state/redisconn.go:55 RedisConn: dial failed {"error": "dial tcp 192.168.0.14:6379: connect: connection refused"}
-
I Redis eventually becomes available, you will get a similar log message:
2020-11-16T21:13:59.829Z DEBUG state/redisconn.go:136 RedisConn is connected
2020-11-16T21:13:59.829Z INFO wec/main.go:120 Redis connection restored, starting server...
2020-11-16T21:13:59.830Z INFO eventstorage/datagrameventstorage.go:34 Trying to connect to unix datagram socket {"unix-datagram": "/home/vagrant/wec_unix_dgram"}
2020-11-16T21:13:59.830Z INFO eventstorage/datagrameventstorage.go:44 Connected to unix datagram socket {"unix-datagram": "/home/vagrant/wec_unix_dgram"}
