Chat now with support
Chat with Support

syslog-ng Premium Edition 6.0.20 - Administrator Guide for syslog-ng Agent for Windows

Chapter 5. Configuring message sources

The syslog-ng Agent for Windows application can read messages from eventlog containers and text files. The following sections explain how to configure these message sources.

Eventlog sources

The syslog-ng Agent for Windows application can collect messages from the standard Windows eventlog containers, as well as from custom containers. The agent automatically forwards the messages from three standard eventlog containers (Application, Security, System). To enable or disable these sources, or to add custom eventlog containers, complete the following steps:

 NOTE:

The syslog-ng Agent for Windows sends its own log messages into the Application eventlog container.

The agent stores the ID of the last message sent to the destination server, so if the agent is not operating for a time (for example it is restarted ), then it starts reading messages from the last stored message ID, sending out all the new messages.

 Caution:

If an eventlog container becomes corrupt, the agent will stop processing the event source. A log message (Eventlog file is corrupt) is sent directly to the log server to notify about the error.

 Caution:

Hazard of data loss! It is not recommended to setup archiving for the event container. It is possible to lose logs if there are non-processed events in the event container when the archiving is started. Windows closes and renames the event container and starts a new one regardless of any reading applications.

To prevent this, enable overwrite events when needed mode in the Windows Event Viewer with the following conditions:

  • The messages are not generated faster than the agent's processing speed.

  • There is enough window between the first and the last events for planned agent stops. Ensure that new events will not overwrite the event last read by the agent during agent stop.

Procedure 5.1. Managing eventlog sources

Figure 5.1. Managing eventlog sources

Managing eventlog sources

Steps: 

  1. Start the configuration interface of the syslog-ng Agent for Windows application.

  2. Select syslog-ng Agent Settings > Eventlog Sources, and double-click on Event Containers.

    • To disable sending messages from an eventlog container, deselect the checkbox before the name of the container.

    • To modify the log facility associated with the messages of the container, select the container, click Edit, and select the log facility to use in the Log Facility field.

  4. Select Apply, then OK. To activate the changes, restart the syslog-ng Agent service.

Procedure 5.2. Adding eventlog sources

Purpose: 

To forward the messages from an eventlog container to your central log server, complete the following steps.

Prerequisites: 

You need to know the name of the eventlog container. If you do not know the name of the container, see Procedure 5.3, “Determining the name of a custom eventlog container” or Procedure 5.4, “Determining the name of a custom eventlog container on Windows XP, or Server 2003”.

Steps: 

  1. Start the configuration interface of the syslog-ng Agent for Windows application.

  2. Select syslog-ng Agent Settings > Eventlog Sources, and double-click on Event Containers.

  3. Click Add, and enter the name of the container into the Event Container Name field. You can use the * and ? wildcard characters in the name of the container. That way you can handle multiple eventlog containers in a single source.

    If you use wildcards in the name of the eventlog container, note the following points:

    • If none of the existing eventlog containers match the pattern, the syslog-ng Agent will send a warning message into the debug log. For details on enabling debug logs, see the section called “Debugging syslog-ng Agent”.

    • The syslog-ng Agent application checks for new eventlog containers only when it starts or restarts. If a new eventlog container is created with a name that matches the pattern of an eventlog source, restart the syslog-ng Agent service.

      		 Caution:

      Hazard of data loss! If you use wildcards in multiple eventlog source names, make sure that only one pattern matches every container name. If two eventlog sources match the same container, syslog-ng Agent might ignore the messages of the eventlog container.

  4. Click Apply, then OK. To activate the changes, restart the syslog-ng Agent service.

    Expected result: 

    The syslog-ng Agent application starts sending new messages from the newly added eventlog container. Note that the syslog-ng Agent will send existing messages from the eventlog container only if you have selected the Read Old Records option.

Procedure 5.3. Determining the name of a custom eventlog container

Purpose: 

To determine the name of a custom eventlog container, complete the following steps.

Steps: 

  1. Open the Event Viewer application.

  2. Select the custom container you are looking for (for example DNS Server).

  3. Right click on the container and select Properties.

  4. The name of the container is the name of the file (without the extension) displayed in the Logname field (for example for C:\WINDOWS\system32\winevt\Logs\Security.evtx it is Security).

  5. Use this name as the name of the custom eventlog container during the procedure described in Procedure 5.1, “Managing eventlog sources”.

    		 NOTE:

    Some containers are not real containers, but show selected messages collected from multiple containers. To forward such messages to the syslog-ng server, you have to find out which real containers are displayed in the container, and add them to the configuration of the syslog-ng Agent.

    Some containers have the %4 characters in their names. When adding these to the syslog-ng Agent, replace %4 with the / (slash) character. For example write microsoft-windows-bits-client/analytic instead of microsoft-windows-bits-client%4analytic.

    If you are sending old messages to the server as well, the syslog-ng Agent will not send the very first message stored in the container. This is a bug in the Windows API.

Procedure 5.4. Determining the name of a custom eventlog container on Windows XP, or Server 2003

Purpose: 

To determine the name of a custom eventlog container on Windows XP, or Server 2003, complete the following steps.

Steps: 

  1. On the client host select Start > Run > regedit.

  2. Navigate to HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Eventlog\. The custom containers are listed here. For example, the following are valid container names: DFS Replication, File Replication Service, DNS Server.

  3. Use this name as the name of the custom eventlog container during the procedure described in Procedure 5.1, “Managing eventlog sources”.

Procedure 5.5. Managing file sources

Purpose: 

The syslog-ng Agent for Windows application can collect log messages from text files. It can process messages spanning multiple lines, and supports the use of wildcards (*, ?) in filenames to be able to follow log files that are automatically rotated. Note that every line of the file that ends with a newline character is considered a separate message. However, if a file contains only a single line that does not end with a newline character, syslog-ng Agent will not process the line.

To configure file sources, complete the following steps:

 Caution:

Files used as file sources must reside locally on the host the syslog-ng Agent application is running on. Files located on network shares are not supported, because the syslog-ng Agent for Windows application is running as a local service and does not have the privileges to access network shares.

 Caution:

If an application deletes a log file, the application must ensure that syslog-ng Agent had enough time to forward the messages from the file to the central server to avoid losing messages.

Example 5.1. Collecting the logs of multiple applications from a single folder

If two applications log into the same folder (for example C:\logs), you have to create two file sources. For example, if the name of the log files is application1-*.log and application2-*.log, respectively, then create two file sources with the C:\logs Base Directory, but with different File Name Filter: application1-*.log and application2-*.log, respectively.

If other applications log into the C:\logs folder, add a separate expression for each application.

By default, the syslog-ng Agent will send every message to the server that arrives into any of the monitored log files.


Figure 5.2. Managing file sources

Managing file sources

Figure 5.3. Sources properties

Sources properties

Steps: 

  1. Start the configuration interface of the syslog-ng Agent for Windows application.

  2. Select syslog-ng Agent Settings > File Sources, double-click on Sources, and check the Enable option.

  3. Select Add > Browse, and select the folder containing the log files in the Base Directory field. Select or enter the name and extension of the log files in the File Name Filter field. Wildcards can be used. The syslog-ng Agent will forward log messages from every file that is located in this folder and has a name that matches the filter expression.

    		 Caution:

    If you use wildcards in multiple file sources, make sure that the files and folders that match the wildcards do not overlap. That is, every file and folder should belong to only one file source. Monitoring a file from multiple wildcard sources can lead to data loss.

    		 Caution:

    Files used as file sources must reside locally on the host the syslog-ng Agent application is running on. Files located on network shares are not supported, because the syslog-ng Agent for Windows application is running as a local service and does not have the privileges to access network shares.

    		 TIP:

    When specifying the Base Directory, you can use the environment variables of Windows, for example %WINDIR%, %SYSTEMROOT%, %PROGRAMFILES%, and so on.

    		 Caution:

    Note that when managing members of a domain, the selected path must be available on the domain members, for example C:\logs must be available on the client hosts and not on the domain controller.

    • To send messages from the files located in the subfolders of the folder set as Base Directory, select the Recursive option.

    • To change the log facility or the log severity associated to the file source, select the desired facility or priority from the Log Facility or Log Severity fields, respectively.

      		 NOTE:

      Significant changes to the settings of a file source can cause the syslog-ng Agent to resend the entire contents of the matching files. This means that log messages already sent earlier to the syslog-ng server may be resent and thus duplicated in the server logs. Configuration changes that can result in such behavior are:

      • changing the Base Directory,

      • changing filter options,

      • changing the Recursive option.

  5. Optional Step: By default, the syslog-ng Agent application starts sending messages from the beginning of the file. If you only want to send the messages that are newly added to the file, deselect the Read Old Records option.

    		 NOTE:

    Be careful when Read Old Records is disabled. If a new file(s) is created while syslog-ng Agent is stopped, the content of this file will not be forwarded, only the new records. To avoid message loss, never disable Read Old Records in the configuration.

  6. Optional Step: By default, the operating system notifies the syslog-ng Agent application when an application modifies a logfile. However, in some cases this does not happen, because the file-monitoring API of Windows does not notice that the file has changed, for example, when monitoring logfiles of the Windows DHCP service.

    In such case, select the Force Directory Polling option. Note that enabling this option decreases the performance of syslog-ng Agent if you monitor lots of logfiles.

  7. By default, the syslog-ng Agent application assumes that the source files are encoded using the default windows ANSI code page, specific to the locale of the host. If the files have a different encoding, select it from the File Encoding field. Note that the log messages are sent to the destinations using UTF-8 encoding.

  8. If a log messages in the log file consists of multiple lines, that is, the log messages contain newline characters, configure syslog-ng Agent to process the related lines as a single message.

    The syslog-ng Agent application can automatically handle Apache Tomcat Catalina and Oracle SQL log messages. To process such messages, select the name of the application from the Multiple Lines > Application field. Note that the timestamp of Tomcat log messages depends on the locale of the host. The syslog-ng Agent for Windows application automatically removes the last CRLF control character from multi-line messages.

    To process multi-line log messages of a different application, complete the following steps.

    1. Select Multiple Lines > Application > Custom, and set the Multiple Lines > Prefix and optionally the Multiple Lines > Garbage fields.

    2. Specify a string or regular expression that matches the beginning of the log messages in the Multiple Lines > Prefix field. If the Prefix option is set, the syslog-ng Agent ignores newline characters from the source until a line matches the regular expression again, and treats the lines between the matching lines as a single message.

      		 NOTE:

      Use as simple regular expressions as possible, because complex regular expressions can severely reduce the rate of processing multi-line messages.

    3. Use the Multiple Lines > Garbage option when processing multi-line messages that contain unneeded parts between the messages. Specify a string or regular expression that matches the beginning of the unneeded message parts. If the Garbage option is set, the syslog-ng Agent ignores lines between the line matching the Garbage expression and the next line matching Prefix.

      When receiving multi-line messages from a source when the Garbage option is set but no matching line is received between two lines that match Prefix, the syslog-ng Agent application will continue to process the incoming lines as a single message until a line matching Garbage is received.

      		 Caution:

      If the Garbage option is set, the syslog-ng Agent application discards lines between the line matching the Garbage and the next line matching Prefix expressions.

    4. Optional Step: After creating and testing a custom pattern, please consider sending your pattern to One Identity so we can include it in a future version of syslog-ng Agent. To share your pattern with One Identity and other syslog-ng Agent users, click Multiple Lines > Send custom pattern to BalaBit. Your e-mail application will open, with an e-mail containing the application name and the pattern.

  9. Select Apply, then OK. To activate the changes, restart the syslog-ng Agent service.

Procedure 5.6. Managing the internal source

Purpose: 

All messages generated internally by syslog-ng Agent for Windows application use the internal source. The syslog-ng Agent for Windows application can forward messages originating from the internal source to certain destinations. To configure the internal source, complete the following steps:

Steps: 

  1. Select syslog-ng Agent Settings and double-click on Global Settings.

  2. Enable Global Settings.

  3. Navigate to Internal Messages.

  4. Select the internal message types to forward to the Application event container, or to Remote destinations (meaning all servers that are configured as normal TCP destinations). The message types correspond to the respective message severities. The default setting is internal error and warning messages forwarded to Application event container, and info messages forwarded to Remote destinations.

    Only the selected message types will be forwarded.

    		 Caution:

    If the same message types are selected for both the Application event container and the Remote destinations, and the application event container is also a source, messages can be duplicated.

    		 NOTE:

    These options will be inherited from GPOs (Group Policy Objects). For details, see the section called “Domain versus local settings”. They can also be exported/imported from an XML configuration also.

  5. Click Apply.

Procedure 5.7. Configuring global settings

Purpose: 

The syslog-ng Agent for Windows application has some global settings that can apply to both eventlog and file sources. To configure the global settings, complete the following procedure:

Steps: 

  1. Start the configuration interface of the syslog-ng Agent for Windows application.

  2. Select syslog-ng Agent Settings and double-click on Global Settings.

  3. Set the default log facility associated to the messages.

  4. By default, the filters and regular expressions (see Chapter 7, Filtering messages) used in the message filters are case-sensitive. To make them case-insensitive, select the Regular Expressions Ignore Case or the Filters Ignore Case options, or both.

    		 NOTE:

    The Regular Expressions Ignore Case option makes the Message Contents filter case-insensitive for both file and eventlog sources. The Filters Ignore Case option makes the Computers, Sources and Categories, and the Users filter case-insensitive.

  5. Select Apply, then OK. To activate the changes, restart the syslog-ng Agent service.

Procedure 5.8. Configuring the hostname format

Purpose: 

The syslog-ng Agent for Windows application can send the hostname macro in different format types (FQDN or short hostname), depending on the domain membership of the host, and the source of the message (eventlog or file). The hostname settings will affect all logs originating from file sources, eventlog sources, as well as MARK messages and internal messages of syslog-ng Agent, for example, start/stop messages.

To prevent using two host licenses from a trusted source, use the same hostname type in every outgoing message.

To determine the hostname, syslog-ng Agent queries the short hostname of the machine at startup, and then attempts to resolve it from the DNS server to receive the FQDN. If DNS resolution is not possible, the hostname will be the short hostname.

 NOTE:

The syslog-ng Agent will never rewrite hostnames.

To configure the hostname format globally, complete the following steps:

Steps: 

  1. Select syslog-ng Agent Settings and double-click on Global Settings.

  2. Enable Global Settings.

  3. Navigate to Hostname.

  4. Select the hostname type to use globally.

    • To use only the short hostname in the $HOST macro of the outgoing message, select Use only hostname. This is the default setting.

      • In case of file sources, MARK messages and internal messages of syslog-ng Agent the outgoing hostname will be the short hostname of the machine.

      • In case of eventlog sources, the hostname will be the short hostname of the event message (for example mypc), or syslog-ng Agent will cut the domain name from the FQDN and use the short hostname part (for example mypc.mycompany.local becomes mypc).

    • To use FQDN (hostname.domain_name) in the $HOST macro of the outgoing message, select Use FQDN.

      • In case of file sources, MARK messages and internal messages of syslog-ng Agent, the hostname will be the FQDN of the machine.

        		 NOTE:

        If there is no DNS server, or the DNS server cannot resolve the hostname, only the simple hostname of the machine will be used.

      • In case of eventlog sources, if the hostname of event message is already an FQDN, syslog-ng Agent will use it as the hostname (for example mypc.mycompany.local will be used as such). If this is not an FQDN, syslog-ng Agent will try to resolve this hostname and use the received FQDN as hostname (for example mypc becomes mypc.mycompany.local).

        		 NOTE:

        If there is no DNS server, or the DNS server cannot resolve the hostname, only the short hostname of the event message will be used.

    • To use a custom domain name that will be appended after the short hostname to receive the FQDN, select Use hostname with custom domain name and enter the domain name to append to the short hostname in the field below. This option affects every outgoing message: eventlog sources, file sources, MARK messages and internal messages of syslog-ng Agent.

      • If the hostname is a short hostname, the custom domain name will be appended after the hostname (for example mypc becomes mypc.customcompany.local).

      • If the hostname is an FQDN, the domain name part will be replaced with the custom domain name (for example if the FQDN in the forwarded message is mypc.mycompany.local and the custom domain name is customcompany.local, the hostname in the outgoing message becomes mypc.customcompany.local).

    		 NOTE:

    The hostname still can be different in the outgoing messages if in the eventlog message, the hostname in the event is different from the machine hostname:

    • In case of a forwarded eventlog: the original machine hostname will be the hostname.

    • The machine hostname is different from what the DNS server provides (if there is a DNS server and it can resolve the hostname).

  5. To use lower-case characters in every hostname, enable Convert to lower-case. This is enabled by default. When disabled, mixed lower-case and upper-case characters (if there is any) will be used in hostnames. This option affects every outgoing message: eventlog sources, file sources, MARK messages and internal messages of syslog-ng Agent.

  6. Click Apply.

Procedure 5.9. Disabling sources and filters globally

Purpose: 

Filters and sources can be disabled globally as well. Disabling filters or sources means that the syslog-ng Agent ignores the disabled settings: that is, if the file sources are disabled, the agent does not send the messages from the files to the server. For details, see the following procedure.

Steps: 

  1. Start the configuration interface of the syslog-ng Agent for Windows application.

    • To disable eventlog sources, select syslog-ng Agent Settings, right-click on Eventlog Sources, then select Properties > Disable.

    • To disable file sources, select syslog-ng Agent Settings, right-click on File Sources, then select Properties > Disable.

    • To disable eventlog filters, select syslog-ng Agent Settings > Destinations, right-click on Global Event Filters, then select Properties > Disable.

    • To disable file filters, select syslog-ng Agent Settings > Destinations, right-click on Global File Filters, then select Properties > Disable.

  3. Select Apply, then OK. To activate the changes, restart the syslog-ng Agent service.

Chapter 6. Using SSL-encrypted connections with the syslog-ng Agent

When connecting to a syslog-ng server using an encrypted connection, the syslog-ng Agent for Windows verifies the certificate of the server. The connection can be established only if the syslog-ng Agent for Windows can verify the certificate of the syslog server. For this, import one of the following certificates into the Certificate Store (MMC > Certificates > Computer Account > Local Computer > Trusted Root Certificates) of the Windows-based host:

  • The certificate of the Certificate Authority (CA) that issued the certificate of the server

  • If your server uses a self-signed certificate, import the self-signed certificate

For details on importing certificates, see Procedure 6.3, “Importing certificates with the Microsoft Management Console”.

 NOTE:

This certificate (sometimes also called the CACert of the server) is not the certificate of the server: it is the certificate of the CA that signed the certificate of the server.

Procedure 6.1. Enabling encrypted connections

Purpose: 

To enable SSL-encrypted connections to the server, complete the following steps:

Steps: 

  1. Start the configuration interface of the syslog-ng Agent for Windows application.

  2. Select syslog-ng Agent Settings > Destinations.

  3. Right-click on the server that accepts encrypted connections and select Properties.

  4. Select the Use SSL option.

    		 Caution:

    The connection is established only if the syslog-ng Agent for Windows can verify the certificate of the syslog server. For this, import one of the following certificates into the Certificate Store (MMC > Certificates > Computer Account > Local Computer > Trusted Root Certificates) of the Windows-based host:

    • The certificate of the Certificate Authority (CA) that issued the certificate of the server

    • If your server uses a self-signed certificate, import the self-signed certificate

    For details on importing certificates, see Procedure 6.3, “Importing certificates with the Microsoft Management Console”.

    		 NOTE:

    The subject_alt_name parameter (or the Common Name parameter if the subject_alt_name parameter is empty) of the server's certificate must contain the hostname or the IP address (as resolved from the syslog-ng clients and relays) of the server (for example syslog-ng.example.com).

    Alternatively, the Common Name or the subject_alt_name parameter can contain a generic hostname, for example *.example.com.

    Note that if the Common Name of the certificate contains a generic hostname, do not specify a specific hostname or an IP address in the subject_alt_name parameter.

  5. Click Advanced Options.

    Figure 6.1. Adding new server

    Adding new server

    To compress the messages during transfer to save bandwidth, select the Allow Compression option. Note that for syslog-ng Agent to actually use compression, the following points must be met.

    • The Server > Advanced Options > Allow Compression option must be enabled.

    • You must use SSL and/or RLTP to send messages to the logserver (that is, at least one of the Use syslog-ng proprietary Reliable Log Transfer Protocol (RLTP) or Use TLS encryption options must be enabled.

    • The logserver must be configured to enable compression. If the logserver is syslog-ng PE the proper allow-compress() option must be enabled in the source. If the logserver is syslog-ng Store Box, enable the Log > Sources > Allow compression option. Note that to send compressed messages to syslog-ng Store Box, you must use the RLTP™ protocol (for details, see the syslog-ng Documentation page).

  6. Select Apply, then OK. To activate the changes, restart the syslog-ng Agent service.

Using mutual authentication with syslog-ng Agent

When the syslog-ng server is configured to use mutual authentication, it requests a certificate from the syslog-ng clients. The syslog-ng Agent application can automatically show the requested certificate to the server when the connection is established, provided it is available in the Personal Certificates store (MMC > Certificates > Computer Account > Local Computer > Personal Certificates) of the Local Computer. Use the Certificate Import Wizard to import this certificate. For details, see Procedure 6.3, “Importing certificates with the Microsoft Management Console”.

 NOTE:

If a certificate revocation list (CRL) is available in the Local Computer/Personal Certificates store, the syslog-ng Agent verifies that the certificate of the syslog-ng server is not on this list.

Procedure 6.2. Configuring mutual authentication with the syslog-ng Agent for Windows

Purpose: 

If the syslog-ng server requests authentication from the syslog-ng Agent, complete the following steps.

Steps: 

  1. Create certificates for the clients. By default, the syslog-ng Agent will look for a certificate that contains the hostname or IP address of the central syslog-ng server in its Common Name. If you use a different Common Name, do not forget to complete Step 3 to set the Common Name of the certificate.

    The certificate must contain the private key and must be in PKCS12 format.

    		 TIP:

    To convert a certificate and a key from PEM format to PKCS12 you can use the following command:

    openssl pkcs12 -export -in agentcertificate.pem -inkey agentprivatekey.pem -out agentcertificatewithkey.pfx

  2. Import this certificate into the Personal Certificate store of the Local Computer using the Certificate Import Wizard. For details, see Procedure 6.3, “Importing certificates with the Microsoft Management Console”.

  3. By default, the syslog-ng Agent will look for a certificate that contains the hostname or IP address of the central syslog-ng server in its Common Name. (The agent will look for the server name or address set in the Server Name field of the destination.) If the certificate of the client has a different Common Name, complete the following steps:

    1. Start the configuration interface of the syslog-ng Agent for Windows application.

    2. Select syslog-ng Agent Settings > Destinations.

    3. Right-click on the server that requires mutual authentication and select Properties.

    4. Select the Use SSL option, click Select, then select the certificate to use. You can also type the Common Name of the certificate into the Client Certificate Subject field.

      If you have more than one certificates with the same Common Name, alternatively, you can type the Distinguished Name (DN) of the certificate into the Client Certificate Subject field. When using the Distinguished Name, type only the elements of the name, separated with comma, starting with the country. For example US, Maryland, Pasadena, Example Inc, Sample Department, mycommonname

      		 NOTE:

      A common way is to use the hostname or the IP address of the host running the syslog-ng Agent as the Common Name of the certificate (for example syslog-ng-agent1.example.com).

  4. Select Apply, then OK. To activate the changes, restart the syslog-ng Agent service.

Procedure 6.3. Importing certificates with the Microsoft Management Console

Purpose: 

To import a certificate, complete the following steps.

Steps: 

  1. Start Microsoft Management Console by executing mmc.exe (Start menu Run application).

    		 NOTE:

    Running mmc.exe requires administrator privileges.

  2. Click on the Add/Remove snap-in item of the File menu.

  3. Click Add, select the Certificates module, and click Add.

  4. Select Computer account in the displayed window and click Next.

  5. Select Local computer and click Close.

  6. To import the CA certificate of the syslog-ng server's certificate, navigate to Console Root > Certificates > Trusted Root Certificate Authorities > Certificates.

    To import a certificate for the syslog-ng Agent to perform mutual authentication, navigate to Console Root > Certificates > Personal > Certificates.

  7. Right-click on the Certificates folder and from the appearing menu select All tasks > Import. The Certificate Import Wizard will be displayed. Click Next.

    Optional step: Certificates used to authenticate the syslog-ng Agent in mutual authentication include the private key. Provide the password for the private key when requested.

  8. Windows offers a suitable certificate store by default, so click Next.

  9. Click Finish on the summary window and Yes on the window that marks the successful importing of the certificate.

Chapter 7. Filtering messages

The syslog-ng Agent for Windows application can filter log messages both in blacklist- and whitelist fashion. When using blacklisting, you can define filters, and any message that matches the filters is ignored by the agent — only messages that do not match the filters are sent to the central server. When using whitelisting, you can define filters, and the messages matching the filters are forwarded to the central server — other messages are ignored. By default, blacklist filtering is used.

If you define multiple filters, the messages must match every filter. In other words, the filters are connected to each other with logical AND operations.

Different filters are available for eventlog- and file sources. When the syslog-ng Agent processes a message, it checks the relevant filters one-by-one: for example if it finds a blacklist filter that matches the message, the agent stops processing the message without sending it to the server.

 NOTE:

By default, all filters are case sensitive. For details on how to change this behavior, see Procedure 5.7, “Configuring global settings”.

Procedure 7.1. Filtering eventlog messages

Purpose: 

The following types of filters are available for eventlog sources. Unless described otherwise, the filters match only if the same string appears in the related field of the message.

 NOTE:

When filtering on the message source, the values of the Source field can be incorrect in some cases. Check the EVENT_SOURCE field of a message to avoid any problems.

  • Sources: Filter on the source (application) that created the message. Corresponds with the EVENT_SOURCE macro.

  • Sources and Event ID: Filter on the source (application) that created the message, and optionally on the identification number of the event. Corresponds with the EVENT_SOURCE and EVENT_ID macros.

  • Message Contents: Filter the text of the message, that is, the contents of the EVENT_MESSAGE macro. In this filter you can use regular expressions.

  • Sources and Categories: Filter on the source (application) that created the message, and optionally on the category of the event. Corresponds with the EVENT_SOURCE and EVENT_CATEGORY macros.

    Example 7.1. Filtering on Sources and Categories

    For example, you want to filter the following message:

    Source: Microsoft Windows security auditing
Category: Process Creation
New Process Name: C:\Windows\System32\SearchProtocolHost.exe

    Set the Source to Microsoft Windows security auditing, and Category to Process Creation.


  • Users: Filter on the username associated with the event. Corresponds with the EVENT_USERNAME macro.

  • Computers: Filter on the name of the computer (host) that created the event. Corresponds with the HOST macro.

  • Event Types: Filter on the type of the event. Corresponds with the EVENT_TYPE macro.

To modify the filters used for eventlog messages, complete the following procedure:

Steps: 

  1. If you want to filter on the source of the message, complete the following steps.

    1. Start the Event Viewer application and find a message from the source that you want to filter.

    2. Select the General tab, and right-click on the value of the Source field.

      Figure 7.1. Finding the name of the Event Source

      Finding the name of the Event Source

    3. Select Copy. Save the saved value somewhere, you will need it later to configure the filter in syslog-ng Agent.

      		 NOTE:

      It is important to use this method, because the actual value of the Source field can be longer than what the Event Viewer displays. (For example, for security messages, the displayed source is often Microsoft Windows security, while the full name of the source is Microsoft Windows security auditing. which includes the dot character at the end.)

      Hovering your mouse over the value of the Source field also displays the full name of the source.

  2. Start the configuration interface of the syslog-ng Agent for Windows application.

    • To apply filters globally to every eventlog message, select syslog-ng Agent Settings > Destinations > Global Event Filters, and right-click Global Event Filters.

    • To apply filters only to a specific destination, select syslog-ng Agent Settings > Destinations, select the destination server, then select Event Filters. Right-click Event Filters.

    		 NOTE:

    If you want to use both global and local (server side) filtering, first global filters will be applied to the eventlog messages and then the local filters.

  4. Select Properties > Enable > OK.

    Figure 7.2. Global event filters

    Global event filters

  5. To use whitelist-filtering, select White List Filtering. By default, syslog-ng Agent uses blacklist filtering.

  6. On the right-hand pane, double-click on the type of filter you want to create.

    • To ignore messages sent by a specific application, or messages of the application with a specific event ID, double-click on Sources and Event ID, select Add, and enter the name of the source (application) whose messages you want to ignore into the Source Name field. To ignore only specific messages of the application, enter the ID of the event into the Event ID field. Select Add > Apply.

      Figure 7.3. Sources and Event ID

      Sources and Event ID

    • To ignore messages that contain a specific string or text, double-click on Message Contents, enter the search term or a POSIX regular expression into the Regular Expression field, then select Add > Apply.

      Figure 7.4. Message Contents

      Message Contents

    • To ignore messages sent by a specific application, or messages of the application that fall into a specific category, double-click on Sources and Categories, select Add, and select the name of the application whose messages you want to ignore from the Application Name field. To ignore only those messages of the application that fall into a specific category, enter the name of the category into the Category field. Select Add > Apply.

    • To ignore messages sent by a specific user, double-click on Users, enter the name of the user into the User field, then select Add > Apply.

    • To ignore messages sent by a specific computer (host), double-click on Computers, enter the name of the user into the Computer field, then select Add > Apply.

    • Event Types: To ignore messages of a specific event-type, double-click on Event Types, select the event types to ignore, and select Ok > Apply.

      		 NOTE:

      Windows labels certain messages as level 3 and the Event Viewer labels such messages as warnings. This is against the official specification: level 3 should not be used, and only level 2 messages are warnings. To filter these events, you have to manually add a new event type to the registry and set its value to 3, for example HKEY_LOCAL_MACHINE\SOFTWARE\BalaBit\syslog-ng Agent\Local Settings\EventSources\Filter\Type\Rule0\Type=3

  8. Select Apply, then OK. To activate the changes, restart the syslog-ng Agent service.

Procedure 7.2. Filtering file messages

Purpose: 

The following types of filters are available for file sources:

  • Message Contents: Filter the text of the message, that is, the contents of the FILE_MESSAGE macro. In this filter you can use regular expressions.

  • File Name: Filter on the file name. Corresponds with the FILE_NAME macro. In this filter you can use wildcards (*, ?). Only available for destination file filters.

To modify the filters used for file messages, complete the following procedure:

Steps: 

  1. Start the configuration interface of the syslog-ng Agent for Windows application.

    • To apply filters globally to every file message, select syslog-ng Agent Settings > Destinations > Global File Filters, and right-click Global File Filters.

    • To apply filters only to a specific destination, select syslog-ng Agent Settings > Destinations, select the destination server, then select File Filters. Right-click File Filters.

    		 NOTE:

    If you want to use both global and local (server side) filtering, first global filters will be applied to the file messages and then the local filters.

  3. Select Properties > Enable > OK.

    Figure 7.5. Global file filters

    Global file filters

  4. To use whitelist-filtering, select White List Filtering. By default, syslog-ng Agent uses blacklist filtering.

  5. On the right-hand pane, double-click on the type of filter you want to create.

    • To ignore messages that contain a specific string or text, double-click on Message Contents, enter the search term or a regular expression into the Regular Expression field, then select Add.

  7. Select Apply, then OK. To activate the changes, restart the syslog-ng Agent service.

Chapter 8. Customizing the message format

The format of the messages received from the eventlog and the file sources can be customized using templates. You can define separate message format for the eventlog and the file sources. If you have multiple destination servers configured, you can define separate templates for each server. When creating a template to customize the message format, you can use macros, all alphanumeric characters, and the following special characters: <>,():;-+/_.

By default, syslog-ng Agent uses the following templates to forward messages:

  • For the BSD protocol: <${PRI}>${BSDDATE} ${HOST} ${MSGHDR}${MESSAGE}

  • For messages read from the eventlog, the $MESSAGE part is ${EVENT_USERNAME}: ${EVENT_NAME} ${EVENT_SOURCE}: [${EVENT_TYPE}] ${EVENT_MSG} (EventID ${EVENT_ID}) for every protocol.

  • For messages read from a file, the $MESSAGE part is $FILE_NAME: $FILE_CURRENT_POSITION/$FILE_SIZE: $FILE_MESSAGE for every protocol.

Procedure 8.1. Customizing messages using templates

Purpose: 

To create a template, complete the following procedure:

 Caution:

These macros are available only in the syslog-ng Agent for Windows. To recognize Windows-specific elements of the log message (for example eventlog-related macros) on the syslog-ng server, you have to use parsers on the syslog-ng server. The parser must be configured to match the message format set in the syslog-ng Agent.

Steps: 

  1. Start the configuration interface of the syslog-ng Agent for Windows application.

  2. Select syslog-ng Agent Settings > Destinations. Select your log server, and click Properties.

  3. To change the format of messages received from eventlog sources, type the message format you want to use into the Event Message Format > Message Template field.

    To change the format of messages received from file sources, type the message format you want to use into the File Message Format > Message Template field.

    Do not forget to add the $ character before macros. For a complete list of the available macros, see the section called “Macros available in the syslog-ng Agent”.

    For example, to send the messages in the DATE HOSTNAME MESSAGE format, type Date:$DATE Hostname:$HOST Logmessage:$MESSAGE.

    Note that the $MESSAGE macro contains not only the text of the log message, but also additional information received from the message source, such as the name of the eventlog container, or the file, as set in the eventlog-specific and file-specific templates.

    		 NOTE:

    Templates are assigned to a single destination server, so it is possible to use different templates for different servers. However, a server and its failover servers always receive the same message.

    		 Caution:

    If you have more than one destination servers configured (separate servers, not in failover mode), and you want to use the same template for every server, you must manually copy the template into the configuration of each server. Template modifications are not applied automatically to every server.

  4. Click OK.

  5. To activate the changes, restart the syslog-ng Agent service.

Customizing the timestamp used by the syslog-ng Agent

The syslog-ng Agent can send the syslog messages using either the ISO or the BSD timestamp format. It is recommended to use the ISO format, because it contains much more information than the BSD format.

Note that in the syslog-ng Agent, the macros without prefix (for example DATE) always refer to the receiving date of the message (for example R_DATE) when it arrived into the event log container, and are included only for compatibility reasons.

 Caution:

If a remote host is logging into the event log of the local host that is running syslog-ng Agent for Windows, both hosts have to be in the same timezone, because the event log message does not include the timezone information of the sender host. Otherwise, the date of the messages received from the remote host will be incorrect.

Related Documents

The document was helpful.

Select Rating

I easily found the information I needed.

Select Rating