Load-Balancing using Syslog-ng Store Box (4238294)

Change the configuration of the Syslog-ng Store Box to "load-balance" the logs to multiple destinations, or a single destination using multiple connections, based upon the millisecond the log was processed within the Syslog-ng Store Box.

For this example the following filters will be used:

"$(% ${R_MSEC} 2)" == "0"
"$(% ${R_MSEC} 2)" == "1"

The number 2 in the above 2 filters means that Modulo (MOD) 2 is being applied, meaning, the log's millisecond section of the processed time stamp will be divided by 2, and the remaining result will be either 0 or 1.

Then, after that division is made and either 0 or 1 is obtained, the filter will check whether it equals 1 or equals 2 and filter the log based upon that.

In this example, load-balancing logs is using 2 filters which means 2 destinations can be used.

To use more than 2 destination simply change the 2 to the number of destinations desired and increase the =="n" (where n is the number previous) by 1. For instance, for 14 destinations the first 3 and last 2 filters would look like the following:

"$(% ${R_MSEC} 14)" == "0"
"$(% ${R_MSEC} 14)" == "1"
"$(% ${R_MSEC} 14)" == "2"
...........................
"$(% ${R_MSEC} 14)" == "12"
"$(% ${R_MSEC} 14)" == "13"

Once the filter statements have been created start with the following steps to implement the load-balancing filter(s) into the SSB:

Step 1.) Log into the SSB's WebUI and navigate to Log > Sources on the left-hand side of the screen.

Step 2.) Create the source(s) required to load-balance against and commit the changes (see example below):

Step 3.) Log into the SSB's WebUI and navigate to Log > Sources on the left-hand side of the screen.

Step 4.) Create the destinations required to load-balance against and commit the changes (see example below):

Step 5.) Log into the SSB's WebUI and navigate to Log > Paths on the left-hand side of the screen.

Step 6.) Create the Log statements using the source created as the source, the external destination as the destination, and then click on the "Custom Filter" link to "drop down" the options where the filter can be applied. Additionally, starting with the first Log Path, enable the "final" flag by checking the checkbox. The last Log Path does not need the "final" flag checked, this is an option, however, all Log Paths previous to the last Log Path defined will need to have the "final" checkbox checked to ensure logs are routed properly. The "final" flag ensures that once the source's logs have been processed by the filter, only the logs the did NOT meet the filter's requirements move onto the next Log Path.

Starting with the first log path, apply the first filter statement, then move onto the second log path and apply the second log statement. (see below example for specific details):

Step 7.) Commit the changes.

After this, incoming logs should become "load-balanced" to the different locations specified in the destination of the Log Paths.