立即与支持人员聊天
与支持团队交流

syslog-ng Premium Edition 7.0.19 - Performance Guideline for syslog-ng Premium Edition

Configuration guidelines

Log messages can be collected and processed at a faster rate in the latest version of syslog-ng Premium Edition compared to version 6 LTS and earlier versions but several configuration aspects will affect the rate at which log messages are collected and stored. The following tables show the results of tests performed with syslog-ng PE version 7.0. Taking the following factors into consideration will optimize syslog-ng Premium Edition performance:

Number of network connections:

In a multithreaded environment, an increase in connections will have no significant impact on the rate at which syslog-ng PE processes log messages.

Table 1: Number of network connections
Number of Connections Messages Per Second Average Data Rate (MB/sec)
10 640,000 240
50 550,000 205
100 530,000 200
200 545,000 205
Configuration: path – TCP, destination – multiple files (using macros), message size: 400 bytes
Encrypted log transfer:

The syslog-ng PE application uses the Transport Layer Security (TLS) protocol to encrypt the communication. TLS also allows the mutual authentication of the host and the server using X.509 certificates.

Table 2: Encrypted log transfer — 10 connections

 

Legacy syslog IETF syslog
Not Encrypted TLS Encryption Not Encrypted TLS Encryption
Messages per second 640,000 620,000 65,000 65,000
Average data rate (MB/sec) 240 230 35 35

Configuration: path – TCP, multithreaded, 10 connections, destination – multiple files (using macros), message size: 400 bytes

Table 3: Encrypted log transfer — 100 connections

 

Legacy syslog IETF syslog
Not Encrypted TLS Encryption Not Encrypted TLS Encryption
Messages per second 565,000 565,000 60,000 60,000
Average data rate (MB/sec) 210 210 30 30

Configuration: path – TCP, multithreaded, 100 connections, destination – multiple files (using macros), message size: 400 bytes

Type of storage:

The syslog-ng PE application can:

Table 4: Type of storage — 10 connections
Type of Storage Messages Per Second Average Data Rate (MB/sec)
Plain text file 270,000 100
Multiple plain text files (using macros, with log messages divided by hostname) 640,000 240
Network destination — legacy syslog 250,000 95
Database destination — MongoDB

In the case of MongoDB destinations, performance is influenced by a number of criteria unrelated to syslog-ng. If you need information on MongoDB-related performance measurements, contact One Identity.

If you are an existing customer, contact our Support Team. Otherwise, contact your Pre-Sales Engineer.

Database destination — SQL

In the case of SQL destinations, performance is influenced by a number of criteria unrelated to syslog-ng. If you need information on SQL-related performance measurements, contact One Identity.

If you are an existing customer, contact our Support Team. Otherwise, contact your Pre-Sales Engineer.

Configuration: path – TCP, multithreaded, 10 connections, message size: 400 bytes

Table 5: Type of storage — 100 connections
Type of Storage Messages Per Second Average Data Rate (MB/sec)
Plain text file 410,000 155
Multiple plain text files (using macros, with log messages divided by hostname) 505,000 190
Network destination — legacy syslog 245,000 90
Database destination — MongoDB

In the case of MongoDB destinations, performance is influenced by a number of criteria unrelated to syslog-ng. If you need information on MongoDB-related performance measurements, contact One Identity.

If you are an existing customer, contact our Support Team. Otherwise, contact your Pre-Sales Engineer.

Database destination — SQL

In the case of SQL destinations, performance is influenced by a number of criteria unrelated to syslog-ng. If you need information on SQL-related performance measurements, contact One Identity.

If you are an existing customer, contact our Support Team. Otherwise, contact your Pre-Sales Engineer.

HDFS 110,000 40

 

 

Note: Processing speed is heavily influenced by the number of HDFS data nodes in use. When syslog-ng writes multiple files to HDFS, and Hadoop places these on different data nodes, then processing speed might increase in proportion to the number of data nodes used (not necessarily in a linear fashion). The data provided here shows performance in the case of a single data node.
Elasticsearch

1,260 (with flush_limit(1))

9,700 (with flush_limit(5000))

1 (with flush_limit(1))

5 (with flush_limit(5000))

Configuration: path – TCP, multithreaded, 100 connections, message size: 400 bytes

Number of files and directories when reading log messages from multiple plain text files:

When reading log messages from a set of files, the number of directories and the number of files per directory used have no significant impact on performance.

Table 6: Number of files and directories — using the inotify monitor method
Number of Directories Number of Files Per Directory Messages Per Second Average Data Rate (MB/sec)
1 1 110,000 45
10 175,000 70
100 150,000 60
10 1 180,000 70
10 150,000 60
100 130,000 50
100 1 150,000 60
10 130,000 50
100 130,000 50
Configuration: path – TCP, multithreaded, monitor-method(inotify), File source message size: 400 bytes
Table 7: Number of files and directories — using the poll monitor method
Number of Directories Number of Files Per Directory Messages Per Second Average Data Rate (MB/sec)
1 1 110,000 45
10 165,000 65
100 150,000 60
10 1 175,000 70
10 150,000 60
100 130,000 50
100 1 150,000 60
10 130,000 50
100 125,000 50
Configuration: path – TCP, multithreaded, monitor-method(poll), File source message size: 400 bytes
Disk buffer:

The syslog-ng Premium Edition stores messages on the local hard disk if the central log server or the network connection to the server becomes unavailable.

Table 8: Disk buffer
Without Disk Buffer Reliable Normal
Messages per second 345,000 40,000 60,000
Average data rate (MB/sec) 130 15 20
Configuration: path – TCP, multithreaded, disk buffer: 1000 MB, 100 connections, message size: 400 bytes
Log pre-processing:

Depending on the type of pre-processing, the rate at which syslog-ng PE collects messages can vary. Rewriting, using parsers, as well as pattern recognition processing through PatternDB have a significant impact on the message processing rate. Regular expressions have only a light impact, while facility filtering and tag filtering have virtually no impact at all.

Note that in a multithreaded environment, PatternDB has a particularly large impact on performance.

When combining multiple types of pre-processing, processing rate will drop below the processing rate of the slowest pre-processing method used.

Table 9: Log pre-processing — 10 connections
Messages Per Second Average Data Rate (MB/sec)
No pre-processing 680,000 255
Simple regexp (for example, matching a single string) 570,000 210
Facility filter 670,000 250
Tag filter 650,000 245
PatternDB (10 % of messages matched) 40,000 15
Simple rewrite (for example, rewrite hostname) 245,000 90
Python parser 35,000 15
JSON parser 40,000 25
kv parser 190,000 70
XML parser 15,000 20
Configuration: path – TCP, multithreaded, 10 connections, message size: 400 bytes
Table 10: Log pre-processing — 100 connections
Messages Per Second Average Data Rate (MB/sec)
No pre-processing 515,000 195
Simple regexp (for example, matching a single string) 510,000 190
Facility filter 500,000 185
Tag filter 530,000 200
PatternDB (10 % of messages matched) 35,000 15
Simple rewrite (for example, rewrite hostname) 360,000 135
Python parser 35,000 15
JSON parser 35,000 25
kv parser 140,000 50
XML parser 15,000 15
Configuration: path – TCP, multithreaded, 100 connections, message size: 400 bytes

The test environment

The test environment consisted of a single client and a server hardware, connected via a Gigabit switch. Note that in certain test runs, the client opened several separate connections to the servers to simulate real-life logging environments. The syslog-ng Premium Edition application was installed from the .run package.

Hardware parameters:

The client hardware had the following main parameters:

  • 2x Intel® Xeon® Processor E5-2620 v3 (15M Cache, 2.40 GHz, 8 GT/s Intel® QPI, 6 cores)

  • Hyperthreading disabled, turbo boost disabled

  • 16 GB RAM

  • 10 Gbps Ethernet

  • HDD 500 GB

  • Operating system: ubuntu-xenial amd64

The server hardware had the following main parameters:

  • 2x Intel® Xeon® Processor E5-2620 v3 (15M Cache, 2.40 GHz, 8 GT/s Intel® QPI, 6 cores)

  • Hyperthreading disabled, turbo boost disabled

  • 16 GB RAM

  • 10 Gbps Ethernet

  • SSD 500 GB

  • Operating system: ubuntu-xenial amd64

Performance improvement:

The following settings were used for performance improvement:

  • Improving performance with lots of connections:

    max_connections = active_connections log_iw_size = number of active_connections * 1000 log_fetch_limit = 1000 flush_lines = 1000 log_fifo_size = log_iw_size * 2 use_dns = no keep_hostname = yes keep_timestamp = no

  • Improving performance with a few connections but high amount of traffic:

    Source side:

    log_iw_size = number of active_connections * 100,000 log_fetch_limit = number of active_connections * 100,000

    Destination side:

    log_fifo_size = max_connections * (log_iw_size/number of active_connections) flush_lines = 10,000 or greater

Resource usage:

The performance tests were carried out in multithreaded mode:

threaded(yes)

One way to optimize the resource usage of syslog-ng PE is to limit the number of worker threads that syslog-ng uses. This helps prevent syslog-ng PE from using all available CPUs. You can limit the number of worker threads using the --worker-threads command-line option that sets the maximum total number of threads syslog-ng PE can use, including the main syslog-ng PE thread.

Note, however, that SQL sources and destinations, as well as Java destinations, such as Elasticsearch, HDFS, and Apache Kafka, always run in their own, separate threads. This means that the --worker-threads command-line option has no impact on them.

相关文档