Return
-
Title
A system service failed. Firmware: core Systemunit: pam-pipeline.service -
Description
In a situation where SPS is setup with a content policy for SSH that is configured to alert based on a credit card rule, In a certain case when an audit trail has data with matched content that is larger than a limit of 32kb, this causes the pam-pipeline.service to fail and the following error can be found in the logs:
[Time Stamp] [SPS.Appliance.Hostname] paa-pipeline[3970963]: ERROR c.b.pam.pipeline.storage.PublisherActor Critical error received, actor system is terminated; reason=Document contains at least one immense term in field="alert.matched_content.raw" (whose UTF8 encoding is longer than the max length 32766), all of which were skipped. Please correct the analyzer to not produce such terms. The prefix of the first immense term is: '[115, 113, 108, 108, 100, 114, 32, 80, 65, 82, 70, 73, 76, 69, 61, 36, 72, 79, 77, 69, 47, 46, 115, 113, 108, 108, 100, 114, 112, 119]...', original message: bytes can be at most 32766 in length; got 32896 [Time Stamp] [SPS.Appliance.Hostname] pam-pipeline[3970963]: 2021-07-29 07:05:34,461 startPipelineSystem-akka.actor.default-dispatcher-8 WARN Unable to register Log4j shutdown hook because JVM is shutting down. Using SimpleLogger [Time Stamp] [SPS.Appliance.Hostname] systemd[1]: health-status.service: Succeeded. [Time Stamp] [SPS.Appliance.Hostname] syslog-ng[184]: Syslog connection closed; fd='22', client='AF_INET(127.0.0.1:48044)', local='AF_INET(127.0.0.1:10514)' [Time Stamp] [SPS.Appliance.Hostname] systemd[1]: pam-pipeline.service: Main process exited, code=exited, status=1/FAILURE [Time Stamp] [SPS.Appliance.Hostname] systemd[1]: pam-pipeline.service: Failed with result 'exit-code'. -
Cause
This issue can impact SPS 6.9.x and 6.10 when matched content is larger than the limit of 32kb. -
Resolution
WORKAROUND:With this workaround, Strings longer than 8191 characters are not analyzed, and they cannot be searched, but they are stored in the content alert event.Upgrading from this 'patched' 6.9.x or 6.10 to 6.11 is safe and supported.Please follow the steps below:1.) Copy the attahced file named "sessions_index.json" to the SPS appliance under this path: /mnt/firmware/root/2.) Login to the core shell of the SPS appliance3.) Issue the following command from the core shell:curl -X PUT "localhost:9200/sessions/_mapping?pretty" -H 'Content-Type: application/json' -d@sessions_index.jsonYou should see the following output from it:{"acknowledged" : true}4.) reboot the applianceRESOLUTION:This issue will be addressed in the next release of SPS version 6.11Previously, if the matched_content field was too large, the publish request failed, which could result in reaching the queue size limit. Strings longer than 8191 characters are not analyzed, and they cannot be searched, but they are stored in the content alert event.This solution is used to protect against Lucene’s term byte-length limit of 32766. -
Change Request
PAM-15051 -
Attachments