Return
-
Title
Systemd journal is too big while trying to upgrade > disk full on boot firmware -
Description
Systemd journal is not being deleted or rotated by the SSB and the file size will increase with time, hence the disk will be full on the boot firmware when trying to upgrade the SSB.
In such cases a pop up window will appear to reboot the SSB. DO NOT REBOOT THE SSB IN SUCH CASE, INSTEAD PERFORM THE APPROPRIATE WORKAROUNDS AS PROVIDED IN THE RESOLUTION FIELD!
-
Cause
Systemd journal can not be removed / rotated by SSB, because the customers can decide to forward the journal logs to an external location, and because we at One Identity don't know the customer's environmental setup.
-
Resolution
One Identity recommends using one of the following workarounds, if facing such issues while trying to upgrade the SSB and if the SSB is running out of boot firmware space.
There are multiple workarounds:
1. If the systemd journal messages are forwarded to either an internal or an external space, those journal messages still on the SSB can be manually removed (as those messages are accessible on the internal / external space).
2. A backup / archive policy can be set up for the systemd journal messages, so those messages will be backed up / archived in an internal / or external space, hence these systemd journal messages can be deleted from the SSB (as those messages are accessible on the backup / archive space).
Please refer to the SSB Admin Guide for further details.
3. To automatically remove the messages from the boot firmware, take the following modifications in conjunction with either item 1 or 2 above to the journal.conf file (/etc/systemd/journald.conf) on the boot firmware:
[Journal]
Storage=persistent
Compress=yes
ForwardToSyslog=yes
SystemMaxUse=150M
MaxRetentionSec=1month
Where the "SystemMaxUse" and the "MaxRetentionSec" can be set to anything else according to the customers requirements (for further setup options, please see the journald manpage).
These modifications will taint the boot firmware (which will appear as an error message once the SSB is about to be updated) in that case please see following KB Article:
https://support.oneidentity.com/kb/4307157/how-to-check-tainted-files-in-the-firmware-how-to-remove-tainted-files
After the modification is done to the journald.conf file, journald must be restarted, so the changes can take affect:
# systemctl restart systemd-journald
NOTE: if the above setup has been initiated in the journal.conf file, please make sure that there is an internal log source defined on the core firmware, or else the boot firmware's internal logs are lost!
-
Additional Information
Please note that newly installed version 7.0.1 has already 4GB boot firmware partition (instead of 2GB in earlier releases).