The monitoring() source allows you to select which statistics of syslog-ng PE you want to monitor. In addition, the statistics are available as structured name-value pairs, so you can format the output similarly to other log messages. That way, you can easily convert the statistics and metrics, for example, into JSON or WELF format. That way, you can send the statistics of your log messages into a monitoring solution.
The monitoring() source queries the statistics (counters) that syslog-ng PE collects, formats them, and optionally resets the counters. The monitoring() source emits only these messages, making it easy to route them to their appropriate destination. The stats-level() global option determines exactly which statistics syslog-ng PE collects.
source s_monitor{ monitoring( query("*") );};
The following configuration increases the stats-level() option to 3, and generates a JSON-formatted message every 10 seconds. The generated message contains every available statistics, and is saved into the /var/log/syslog-ng-statistics.log file.
@version: 7.0 options{ stats-level(3); keep-hostname(no); }; source s_monitor{ monitoring( query("*") freq(10) message-template('$(format-json --scope nv_pairs)') );}; destination d_file { file("/var/log/syslog-ng-statistics.log"); }; log { source(s_monitor); destination(d_file); };
The generated message is similar to this one:
[2017-04-03T14:00:31.786133] Outgoing message; message='Apr 3 14:00:31 example-hostname syslog-ng[12281]: {"src":{"severity":{"7":{"processed":"0"},"6":{"processed":"0"},"5":{"processed":"0"},"4":{"processed":"0"},"3":{"processed":"0"},"2":{"processed":"0"},"1":{"processed":"0"},"0":{"processed":"0"}},"monitoring":{"s_monitor#0":{"stamp":"0","processed":"0"}},"facility":{"other":{"processed":"0"},"9":{"processed":"0"},"8":{"processed":"0"},"7":{"processed":"0"},"6":{"processed":"0"},"5":{"processed":"0"},"4":{"processed":"0"},"3":{"processed":"0"},"23":{"processed":"0"},"22":{"processed":"0"},"21":{"processed":"0"},"20":{"processed":"0"},"2":{"processed":"0"},"19":{"processed":"0"},"18":{"processed":"0"},"17":{"processed":"0"},"16":{"processed":"0"},"15":{"processed":"0"},"14":{"processed":"0"},"13":{"processed":"0"},"12":{"processed":"0"},"11":{"processed":"0"},"10":{"processed":"0"},"1":{"processed":"0"},"0":{"processed":"0"}}},"source":{"s_monitor":{"processed":"0"}},"global":{"sdata_updates":{"processed":"0"},"payload_reallocs":{"processed":"2"},"msg_clones":{"processed":"0"}},"destination":{"d_file":{"processed":"0"}},"center":{"received":{"processed":"0"},"queued":{"processed":"0"}},"PROGRAM":"syslog-ng","PID":"12281"}\x0a'
For reference, the JSON part in a readable format is:
{ "center" : { "queued" : { "processed" : "0" }, "received" : { "processed" : "0" } }, "destination" : { "d_file" : { "processed" : "0" } }, "global" : { "msg_clones" : { "processed" : "0" }, "payload_reallocs" : { "processed" : "2" }, "sdata_updates" : { "processed" : "0" } }, "PID" : "12281", "PROGRAM" : "syslog-ng", "source" : { "s_monitor" : { "processed" : "0" } }, "src" : { "facility" : { "0" : { "processed" : "0" }, "1" : { "processed" : "0" }, "2" : { "processed" : "0" }, "3" : { "processed" : "0" }, "4" : { "processed" : "0" }, "5" : { "processed" : "0" }, "6" : { "processed" : "0" }, "7" : { "processed" : "0" }, "8" : { "processed" : "0" }, "9" : { "processed" : "0" }, "10" : { "processed" : "0" }, "11" : { "processed" : "0" }, "12" : { "processed" : "0" }, "13" : { "processed" : "0" }, "14" : { "processed" : "0" }, "15" : { "processed" : "0" }, "16" : { "processed" : "0" }, "17" : { "processed" : "0" }, "18" : { "processed" : "0" }, "19" : { "processed" : "0" }, "20" : { "processed" : "0" }, "21" : { "processed" : "0" }, "22" : { "processed" : "0" }, "23" : { "processed" : "0" }, "other" : { "processed" : "0" } }, "monitoring" : { "s_monitor#0" : { "processed" : "0", "stamp" : "0" } }, "severity" : { "0" : { "processed" : "0" }, "1" : { "processed" : "0" }, "2" : { "processed" : "0" }, "3" : { "processed" : "0" }, "4" : { "processed" : "0" }, "5" : { "processed" : "0" }, "6" : { "processed" : "0" }, "7" : { "processed" : "0" } } } }
The monitoring() driver has the following options. Only the query() option is required, other options are optional.
Type: | boolean |
Default: | no |
Description: Reset the counters after reading. Note that if a destination is not available, syslog-ng PE will not reset its counter even if clear-on-read() is set to yes.
If you use multiple monitoring source, and you use the clear-on-read() parameter, make sure to adjust the queries appropriately. Overlapping queries that read and reset the same counters result in incorrect statistics.
Type: | integer |
Default: | 600 [seconds] |
Description: Specifies how often does syslog-ng PE execute the query and send a statistics message.
Type: | string |
Default: | N/A |
Description: Specifies how the message containing the queried statistics is formatted. You can use macros and template functions in the format string. For example, you can format the message as a JSON object:
source s_monitor{ monitoring( query("*") freq(10) message-template('$(format-json --scope nv_pairs)') );};
Note that here you can only format the payload of the message (that is the, ${MESSAGE} part). You can format the headers or other parts of the outgoing message in the destination driver.
Type: | string |
Default: | N/A |
Description: Specifies which statistical counters will be included in the messages. Note that the list of available counters depends on your syslog-ng PE configuration (mainly the configured sources and destinations) and on the stats-level() global option. The * string includes every available counters. The syntax of the query option is identical to the syslog-ng-ctl query get <query> command.
source s_monitor{ monitoring( query("*") );};
For example, the "destination*" query lists the configured destinations, and the metrics related to each destination. An example output:
destination.java.d_elastic#0.java_dst(ElasticSearch,elasticsearch-syslog-ng-test,t7cde889529c034aea9ec_micek).stats.dropped: 0 destination.java.d_elastic#0.java_dst(ElasticSearch,elasticsearch-syslog-ng-test,t7cde889529c034aea9ec_micek).stats.processed: 0 destination.java.d_elastic#0.java_dst(ElasticSearch,elasticsearch-syslog-ng-test,t7cde889529c034aea9ec_micek).stats.stored: 0 destination.d_elastic.stats.processed: 0
This source is actually preconfigured monitoring() source that generates statistics messages in WELF format. Starting with
@version: 7.0 @include 'scl.conf' options { stats_level(3); }; source s_monitoring_welf { monitoring-welf(freq(10) query('*')); }; destination d_file { file("/tmp/output.txt"); }; log { source(s_monitoring_welf); destination(d_file); };
The output is similar to the following:
Apr 3 14:03:26 example-host syslog-ng[12363]: PID=12363 PROGRAM=syslog-ng center.queued.processed=0 center.received.processed=0 destination.d_file.processed=0 global.msg_clones.processed=0 global.payload_reallocs.processed=2 global.sdata_updates.processed=0 source.s_monitoring_welf.processed=0 src.facility.0.processed=0 src.facility.1.processed=0 src.facility.10.processed=0 src.facility.11.processed=0 src.facility.12.processed=0 src.facility.13.processed=0 src.facility.14.processed=0 src.facility.15.processed=0 src.facility.16.processed=0 src.facility.17.processed=0 src.facility.18.processed=0 src.facility.19.processed=0 src.facility.2.processed=0 src.facility.20.processed=0 src.facility.21.processed=0 src.facility.22.processed=0 src.facility.23.processed=0 src.facility.3.processed=0 src.facility.4.processed=0 src.facility.5.processed=0 src.facility.6.processed=0 src.facility.7.processed=0 src.facility.8.processed=0 src.facility.9.processed=0 src.facility.other.processed=0 src.monitoring.s_monitoring_welf#0.processed=0 src.monitoring.s_monitoring_welf#0.stamp=0 src.severity.0.processed=0 src.severity.1.processed=0 src.severity.2.processed=0 src.severity.3.processed=0 src.severity.4.processed=0 src.severity.5.processed=0 src.severity.6.processed=0 src.severity.7.processed=0\x0a'
Starting with version
© 2021 One Identity LLC. ALL RIGHTS RESERVED. Feedback 使用条款 隐私