| Type: | number (bytes) |
| Default: | 536870912 |
Description: If the size of memory (in bytes) required by journal files increases above this value, syslog-ng PE maps only a single block of every logstore journal into the memory. Default value: 536870912 (512 MB).
If the memory required for the journal files exceeds the logstore-journal-shmem-threshold() limit, syslog-ng PE will store only a single journal block of every journal file in the memory, and — if more blocks are needed for a journal — store the additional blocks on the hard disk. Opening new logstore files means allocating memory for one new journal block for every new file. In extreme situations involving large traffic, this can lead to syslog-ng PE consuming the entire memory of the system. Adjust the journal-block-size() and your file-naming conventions as needed to avoid such situations. For details on logstore journals, see Journal files.
Example: Calculating memory usage of logstore journals
If you are using the default settings (4 journal blocks for every logstore journal, one block is 1MB, logstore-journal-shmem-threshold() is 512MB), this means that syslog-ng PE will allocate 4MB memory for every open logstore file, up to 512MB if you have 128 open logstore files. Opening a new logstore file would require 4 more megabytes of memory for journaling, bringing the total required memory to 516MB, which is above the logstore-journal-shmem-threshold(). In this case, syslog-ng PE switches to storing only a single journal block in the memory, lowering the memory requirements of journaling to 129MB. However, opening more and more logstore files will require more and more memory, and this is not limited, except when syslog-ng PE reaches the maximum number of files that can be open (as set in the --fd-limit command-line option).
Example: Limiting the memory use of journal files
The following example causes syslog-ng PE to map only a single journal block into the host's memory if the total memory range used by logstore journals would be higher than 32 MB.
options {
logstore-journal-shmem-threshold(33554432);
};
destination d_messages { logstore("/var/log/messages_logstore.lgs"
journal-block-size(19660800)
journal-block-count(5)
);
};