One Identity Safeguard for Privileged Sessions 7.4

Reindexing lucene indices enables you to use reindexed indices with Elasticsearch.

NOTE: Reindexing can be very time consuming. To avoid unwanted reindexing, see the following examples below for setting time and initiating dry start.

Prerequisite

Sessions and audit trails are still available (were not cleaned up).
Make sure you have SPS version 6.0 (or later).

To reindex lucene indices

Create and run the following Python script from the core shell.

#!/usr/bin/env python

from datetime import datetime

import os.path

content_store_changed_filename="/opt/scb/var/upgrade/content-store-changed-time/content_store_changed_timestamp"

if not os.path.isfile(content_store_changed_filename):

 print("The "+content_store_changed_filename+" file does not exist. This file contains the information when lucene storage was updated to elastic search.")

 exit()

f=open(content_store_changed_filename)

s=float(f.read())

f.close()

my_date =  datetime.fromtimestamp(s)

print("Run this command to reindex everything before lucene->elasticsearch update:")

print("indexerctl reindex query \"*\" --end "+my_date.isoformat())

You will receive a result similar to this:

Example

indexerctl reindex query "*" --end 2021-05-22T20:21:05.604341

The example shows that all items will be reindexed until 2021-05-22 20:21:05.

You can set the start time parameter and check the list of items scheduled for reindexing:
Example
indexerctl reindex query "*" --end 2021-05-22T20:21:05.604341 --dry --start 2021-05-22T18:21:05.604341
The example shows that all indices will be reindexed between 2021-05-22 18:21:05 and 2021-05-22 20:21:05.

If you are satisfied with the results, start reindexing:

Example

indexerctl reindex query "*" --end 2021-05-22T20:21:05.604341 --start 2021-05-22T18:21:05.604341

Configuring the internal indexer

This section describes how to configure One Identity Safeguard for Privileged Sessions (SPS) to index the audit trails.

Indexing is a resource intensive (CPU and hard disk) operation, and depending on the number of processed audit trails and parallel connections passing SPS, may affect the performance of SPS. Test it thoroughly before enabling it in a production environment that is under heavy load. If your SPS appliance cannot handle the connections and the indexing, consider using external indexers (see ) to decrease the load on SPS. For sizing recommendations, ask your One Identity partner or contact our Support Team.

Note that the minimum value of Backup & Archive > Archive policies > Delete data from SPS after is 30 days when using the indexer service. If you previously had a setting lower than this, it will still archive the index after 30 days when the indexer service is used.

NOTE: Only those audit trails will be processed that were created after full-text indexing had been configured for the connection policy. It is not possible to process already existing audit trails.

NOTE: Using content policies significantly slows down connections (approximately 5 times slower), and can also cause performance problems when using the indexer service.

NOTE: The version of the external indexer must be equal to or greater than the version of One Identity Safeguard for Privileged Sessions (SPS). To make sure you meet this criterion, One Identity recommends that you always upgrade your external indexer when you upgrade SPS. You can check that SPS has established a connection to the external indexer on the Indexer > Worker status page of the SPS web interface.

To configure SPS to index the audit trails

Navigate to Basic Settings > Local Services > Indexer service.

Figure 262: Basic Settings > Local Services > Indexer service > Configure the Indexer service of SPS
Define the Maximum number of parallel audit trails to index on box.

This option determines the maximum number of parallel indexing tasks that the SPS appliance performs. The default value is set to the number of detected CPU cores. Note that indexing audit trails requires about 50-100 Mbytes of memory for terminal sessions (SSH, Telnet, TN3270), and 150-300 Mbytes for graphical sessions (RDP, ICA, VNC, X11). Consider the memory usage of your SPS host before modifying this value.
Define the Maximum number of parallel audit trails to index near real-time on box.

This option determines the maximum number of parallel indexing tasks that the SPS appliance performs near real-time, meaning that indexing starts when sessions are still ongoing. The default value is set to 0.

NOTE: A connection policy configured with near real-time priority (Connection policy > Enable indexing > Priority) requires that you set Maximum number of parallel audit trails to index near real-time on box to a value other than 0.
(Optional) If you have encrypted audit trails and you want to index them, upload the necessary RSA private keys.

Click , and then click the icon to upload a private key. A pop-up window is displayed.

Select Browse, select the file containing the private key, provide the Password if the key is password-protected, and click Upload. Alternatively, you can also copy-paste the private key into the Key field, provide the Password there, and click Set.

TIP: If you want to search in the trail content on the web interface: to view screenshots generated from encrypted audit trails, you also have to upload the necessary private encryption keys to your audit keystore. For more information, see Audit keystore.
Click .
Navigate to Policies > Indexer Policies.
Two Indexer Policies are available by default, both with automatic language detection:
- full_indexing: Slower, indexes the complete content of the screen, including all events.
- lightweight_indexing: Significantly faster, but it extracts only the executed commands (Command event) and the window titles (Window title event) that appear on the screen. It does not index any other screen content (for example, text that is displayed in a terminal or that appears in an RDP window).
  
  For example, in the case of an SSH protocol, lightweight_indexing will index a command with parameters, such as cat --help, but will not index terminal printouts such as the help content itself.
  
  When you add a new Connection Policy, the lightweight_indexing Indexer Policy is assigned to it by default.
NOTE: In the case of graphical protocols, the default Optical Character Recognition (OCR) configuration is automatic language detection. This means that the OCR engine will attempt to detect the languages of the indexed audit trails automatically. However, if you know in advance what language(s) will be used, create a new Indexer Policy.

To create a new Indexer Policy, click .

Figure 263: Policies > Indexer Policies > Indexing options and manual language selection
Select from the indexing options as follows:
- Commands: Allows you to browse, search, and analyze the commands issued in SSH and Telnet sessions.
  
  Caution:
  Do not disable unless you have a reason to do so.
- Window titles: Text appearing as window titles that can be detected on the screen in RDP, Citrix ICA, and VNC connections. Window title detection involves Optical Character Recognition (OCR) on parts of the screen, and can be slightly resource-intensive. SPS versions up till 6.2 only detected only the active window in the screen. From SPS version 6.3, multiple windows can be detected.
  Limitations
  - Default Windows themes are supported.
  - Windows that do not have an X (close window) button in the top-right corner (or it is not visible) are not detected.
  - Use window title detection for sessions that use a single monitor. The feature works in multi-monitor environments as well, but becomes very slow, therefore it is not recommended.
  - Window title detection is case-insensitive.
  Caution:
  Do not disable unless you have a reason to do so.
- Full screen contents: Select this option if you want to search in the full screen content of the audited sessions.
  
  Extract every text that appears on the screen in Telnet, SSH, RDP, Citrix ICA or VNC connections. For graphical protocols, extracting screen content involves Optical Character Recognition (OCR) that requires lots of CPU and disk (consider installing external indexers).
- Pointing device biometrics: Select this option only if you are using One Identity Safeguard for Privileged Analytics (SPA)).
  
  Extract biometric data to authenticate the users based on their pointing device (for example, mouse) usage patterns. SPA can analyze mouse movement patterns of your users as a biometric identity verification method to protect against account theft.
- Typing biometrics: Select this option only if you are using One Identity Safeguard for Privileged Analytics (SPA)).
  
  Extract biometric data to authenticate the users based on their typing dynamics. SPA can analyze the typing patterns of your users as a biometric identity verification method to protect against account theft.
To configure what languages to detect, select Select languages manually for character recognition. Select the language(s) to detect. Note the following:
- Specifying only one language provides the best results in terms of performance and precision.
- The English language is always detected along with the non-English languages that you have configured. However, if you want the OCR to only recognize the English language, you have to select it from the list of languages.
- There are certain limitations in the OCR engine when recognizing languages with very different character sets. For this reason, consider the following:
  - When selecting Asian languages (Simplified Chinese, Traditional Chinese, Korean), avoid adding languages that use the Latin alphabet.
  - When selecting the Arabic language, avoid selecting any other languages.
  - The Thai language is currently not supported. If you are interested in using SPS to index Thai texts, contact our Sales Team.
Specify an accuracy level for Optical Character Recognition (OCR). Each accuracy level brings a different degree of speed and accuracy:
- Fast: The fastest option with potentially less accurate results. Select this option if speed is more important to you than getting the most accurate results possible.
- Balanced (default setting): Fairly accurate option with less than optimum speed. Select this option if you want results to be fairly accurate but you have more than a few sessions to process and processing time is less of a concern.
- Accurate: The most accurate option with less optimal speed. Select this option if you must have the most accurate results possible and speed is less important or you only have a few sessions to process.
Configure the Indexing policy for the Connection policy that you want to index:

By default, the lightweight_indexing Indexing policy is enabled for every Connection policy with normal priority. If this is ideal for you, skip this step and continue with the next step. If you want to use a different policy, for example because you want to OCR the complete screen content, or because you have created a language-specific indexer policy, complete the following substeps.
1. Navigate to the Control > Connections page of the traffic type (for example SSH Control), and select the connection policy to index.
2. Figure 264: <Protocol name> Control > Connections > Enable indexing — Select Indexing Policy
  
  Select the Indexing Policy to be used. Both built-in Indexer Policies feature automatic language detection. To specify a particular language detection configuration, select the Indexing Policy you have created before (in Step 6).
3. To determine the priority level of indexing this connection, select the appropriate Priority level. Selecting a high priority level means that the trails of this connection will be indexed first. Selecting a low priority level means that the trails of this connection will be indexed also, but there might be a delay in indexing if there are a lot of high-priority connections waiting to be indexed. Selecting near real-time means that the indexing of sessions starts when sessions are still ongoing.
4. Click .
Check which channel policy is used in the connection, and navigate to the <Protocol name> Control > Connections page. Select the channel policy used in the connection to index.
On the <Protocol name> Control > Channel Policies page, verify that the Record audit trail option is selected for the channels you want to index (for example, the Session shell channel in SSH, or the Drawing channel in RDP).
Click .

TIP: To verify that indexing works as configured, start a session that uses this connection policy (connect from a client to a server).

When the session is finished, navigate to the Indexer > Indexer status page to verify that the indexer service is processing the audit trail.

If the audit trails are encrypted, ensure that the required decryption keys have been uploaded to Basic Settings > Local Services > Indexer service > Indexer keys.

Configuring external indexers

If One Identity Safeguard for Privileged Sessions (SPS) audits lots of connections, processing and indexing the created audit trails requires significant computing resources, which may not be available in the SPS appliance. To decrease the load on the SPS appliance, you can install the indexer service on external Linux hosts. These external indexer hosts run the same indexer service as the SPS appliance, and can index audit trails, or generate screenshots and replayable video files from the audit trails as needed. The external indexers register on SPS, wait for SPS to send an audit trail to process, process the audit trail, then return the processed data to SPS. The external indexer hosts do not store any data, thus any sensitive data is available on the host while it is being processed.

To use external indexers to process your audit trails, you have to complete the following steps.

Read the conditions and limitations related to external indexers in Prerequisites and limitations.
Install and configure the hosts (physical or virtual) that will run the external indexer service. For details on the hardware requirements, see Hardware requirements for the external indexer host.
Configure SPS to use external indexers. For details, see Configuring One Identity Safeguard for Privileged Sessions (SPS) to use external indexers.
Install and configure the indexer application on the external hosts. For details, see Installing the external indexer Configuring the external indexer.
If you enabled audit trail encrypting on SPS, you will also need to upload the necessary certificates to the external indexer to allow indexing the encrypted trails. For details, Uploading decryption keys to the external indexer.