Version 6.0.20 of the syslog-ng Premium Edition (syslog-ng PE) application is supported on AIX 7.1 for PowerPC.
Preface
Target audience and prerequisites
Products covered in this guide
Typographical conventions
About this document
Chapter 1. Introduction to syslog-ng
What syslog-ng is
What syslog-ng is not
Why is syslog-ng needed?
Secure and reliable log transfer
Flexible data extraction and processing
Chapter 1. Introduction to syslog-ng
Chapter 1. Introduction to syslog-ng
Chapter 1. Introduction to syslog-ng
What is new in syslog-ng Premium Edition 6 LTS?
Who uses syslog-ng?
Supported platforms
Chapter 2. The concepts of syslog-ng
Logging with syslog-ng
Modes of operation
Global objects
Timezones and daylight saving
Versions and releases of syslog-ng PE
Licensing
GPL and LGPL licenses
High availability support
The structure of a log message
Message representation in syslog-ng PE
Structuring macros, metadata, and other value-pairs
Things to consider when forwarding messages between syslog-ng PE hosts
NFS file system for log files
Chapter 3. Installing syslog-ng
Security-enhanced Linux: grsecurity, SELinux
Installing syslog-ng using the .run installer
Upgrading syslog-ng PE
Uninstalling syslog-ng PE
Chapter 4. The syslog-ng PE quick-start guide
Chapter 5. The syslog-ng PE configuration file
The configuration syntax in detail
Notes about the configuration syntax
Global and environmental variables
Logging configuration changes
Modules in syslog-ng PE
Managing complex syslog-ng configurations
Chapter 6. Collecting log messages — sources and source drivers
Collecting internal messages
Collecting messages from text files
Collecting messages using the RFC3164 protocol (network() driver)
Collecting messages from named pipes
Receiving messages from external applications
Collecting messages from tables or relational database
Collecting messages on Sun Solaris
Collecting messages using the IETF syslog protocol (syslog() driver)
Collecting the system-specific log messages of a platform
Collecting messages from the systemd-journal system log storage
Collecting messages from remote hosts using the BSD syslog protocol
Collecting systemd messages using a socket
Collecting messages from UNIX domain sockets
Chapter 7. Sending and storing log messages — destinations and destination drivers
Sending messages directly to Elasticsearch version 2.0 or higher
Storing messages in plain-text files
Storing messages in encrypted files
Storing messages in a MongoDB database
Sending messages to a remote log server using the RFC3164 protocol (network() driver)
Sending messages to named pipes
Sending messages to external applications
Sending SNMP traps
Sending messages to a remote log server using the IETF-syslog protocol
Sending messages to a remote log server using the legacy BSD-syslog protocol (tcp(), udp() drivers)
Sending messages to UNIX domain sockets
Sending messages to a user terminal — usertty() destination
Chapter 8. Routing messages: log paths, reliability, and filters
Managing incoming and outgoing messages with flow-control
Using disk-based and memory buffering
Client-side failover
Filters
Dropping messages
Chapter 9. Global options of syslog-ng PE
Chapter 10. TLS-encrypted message transfer
Chapter 12. Reliable Log Transfer Protocol™
Chapter 13. Reliability and minimizing the loss of log messages
Flow control, no disk buffering, no RLTP™
Flow control, normal disk buffering, no RLTP™
Flow control, reliable disk buffering, no RLTP™
Flow control, reliable disk buffering, RLTP™
Deciding which loss prevention mechanism to apply
Chapter 14. Manipulating messages
Chapter 15. Parsing and segmenting structured messages
Chapter 16. Processing message content with a pattern database
Using pattern databases
Correlating log messages
Triggering actions for identified messages
Creating pattern databases
Chapter 17. Statistics and metrics of syslog-ng
Chapter 18. Multithreading and scaling in syslog-ng PE
Chapter 19. Troubleshooting syslog-ng
Collecting debugging information with strace, truss, or tusc
Stopping syslog-ng
Reporting bugs and finding help
Recover data from orphaned diskbuffer files
Chapter 20. Best practices and examples
GNU General Public License v2
Supported platforms
Chapter 2. The concepts of syslog-ng
Chapter 2. The concepts of syslog-ng
Table of Contents
- The philosophy of syslog-ng
- Logging with syslog-ng
- Modes of operation
- Logging with syslog-ng
- Global objects
- Timezones and daylight saving
- Versions and releases of syslog-ng PE
- Licensing
- GPL and LGPL licenses
- High availability support
- The structure of a log message
- High availability support
- Message representation in syslog-ng PE
- Structuring macros, metadata, and other value-pairs
- Things to consider when forwarding messages between syslog-ng PE hosts
- NFS file system for log files
This chapter discusses the technical concepts of syslog-ng.
Typically, syslog-ng is used to manage log messages and implement centralized logging, where the aim is to collect the log messages of several devices on a single, central log server. The different devices — called syslog-ng clients — all run syslog-ng, and collect the log messages from the various applications, files, and other sources. The clients send all important log messages to the remote syslog-ng server, which sorts and stores them.
Logging with syslog-ng
The syslog-ng application reads incoming messages and forwards them to the selected destinations. The syslog-ng application can receive messages from files, remote hosts, and other sources.
Log messages enter syslog-ng in one of the defined sources, and are sent to one or more destinations.
Sources and destinations are independent objects, log paths define what syslog-ng does with a message, connecting the sources to the destinations. A log path consists of one or more sources and one or more destinations: messages arriving from a source are sent to every destination listed in the log path. A log path defined in syslog-ng is called a log statement.
Optionally, log paths can include filters. Filters are rules that select only certain messages, for example, selecting only messages sent by a specific application. If a log path includes filters, syslog-ng sends only the messages satisfying the filter rules to the destinations set in the log path.
Other optional elements that can appear in log statements are parsers and rewriting rules. Parsers segment messages into different fields to help processing the messages, while rewrite rules modify the messages by adding, replacing, or removing parts of the messages.
Procedure 2.1. The route of a log message in syslog-ng
Purpose:
The following procedure illustrates the route of a log message from its source on the syslog-ng client to its final destination on the central syslog-ng server.
Steps:
-
A device or application sends a log message to a source on the syslog-ng client. For example, an Apache web server running on Linux enters a message into the
/var/log/apachefile. -
The syslog-ng client running on the web server reads the message from its
/var/log/apachesource. -
The syslog-ng client processes the first log statement that includes the
/var/log/apachesource. -
The syslog-ng client performs optional operations (message filtering, parsing, and rewriting) on the message, for example, it compares the message to the filters of the log statement (if any). If the message complies with all filter rules, syslog-ng sends the message to the destinations set in the log statement, for example, to the remote syslog-ng server.
Caution: Message filtering, parsing, and rewriting is performed in the order that the operations appear in the log statement.
NOTE: The syslog-ng client sends a message to all matching destinations by default. As a result, a message may be sent to a destination more than once, if the destination is used in multiple log statements. To prevent such situations, use the
finalflag in the destination statements. For details, see Table 8.1, “Log statement flags”. -
The syslog-ng client processes the next log statement that includes the
/var/log/apachesource, repeating Steps 3-4. -
The message sent by the syslog-ng client arrives from a source set in the syslog-ng server.
-
The syslog-ng server reads the message from its source and processes the first log statement that includes that source.
-
The syslog-ng server performs optional operations (message filtering, parsing, and rewriting) on the message, for example, it compares the message to the filters of the log statement (if any). If the message complies with all filter rules, syslog-ng sends the message to the destinations set in the log statement.
Caution: Message filtering, parsing, and rewriting is performed in the order that the operations appear in the log statement.
-
The syslog-ng server processes the next log statement, repeating Steps 7-9.
NOTE: The syslog-ng application can stop reading messages from its sources if the destinations cannot process the sent messages. This feature is called flow-control and is detailed in the section called “Managing incoming and outgoing messages with flow-control”.
Modes of operation
The syslog-ng Premium Edition application has three distinct operation scenarios: Client, Server, and Relay. The syslog-ng PE application running on a host determines the mode of operation automatically based on the license and the configuration file.
In client mode, syslog-ng collects the local logs generated by the host and forwards them through a network connection to the central syslog-ng server or to a relay. Clients often also log the messages locally into files.
No license file is required to run syslog-ng in client mode.
In relay mode, syslog-ng receives logs through the network from syslog-ng clients and forwards them to the central syslog-ng server using a network connection. Relays also log the messages from the relay host into a local file, or forward these messages to the central syslog-ng server.
You cannot use the following destinations in relay mode: mongodb(), pipe(), sql(). The file() and logstore() destinations work only for local messages that are generated on the relay.
No license file is required to run syslog-ng in relay mode.
In server mode, syslog-ng acts as a central log-collecting server. It receives messages from syslog-ng clients and relays over the network, and stores them locally in files, or passes them to other applications, for example log analyzers.
Running syslog-ng Premium Edition in server mode requires a license file. The license determines how many individual hosts can connect to the server. For details on how syslog-ng PE calculates the number of hosts, see the section called “Licensing”.