Configuring a service pool
You can share your worker resources between multiple indexer services, for example behind a load-balancer, or in an SPS cluster if you want to share some of your indexer workers between multiple SPS instances.
To enable resource sharing
-
Use the indexer-box-config script with an additional --service-pool option to enumerate the service configurations.
NOTE: Running this script can overwrite your custom modifications in your indexerworker.cfg file. Make a backup of indexerworker.cfg before running the script.
indexer-box-config
cp /etc/indxer/indexerworker.cfg /etc/indxer/indexerworker.cfg.bak |
indexer-box-config $dedicated-server-config --service_pool $cfg1 $cfg2 $cfg3 |
If you run the script with --service-pool option, the script creates a service-pool field in your configuration file which can be used by any worker-group. The script also creates a default worker-group called shared, with an additional balancing field. These workers run in one-shot mode, and connect to an indexer-service randomly selected from the pool.
Example: output after running script with two configuration files
{ |
"service": { |
"address": "169.254.1.1", |
"port": 12345, |
"ssl": { |
"ca_certificate": "/etc/indexer/cacert.pem", |
"certificate": "/etc/indexer/worker.pem", |
"enabled": false, |
"private_key": "/etc/indexer/worker.key" |
} |
}, |
"service_pool": [ |
{ |
"address": "192.168.1.111", |
"port": 12345, |
"ssl": { |
"ca_certificate": "/etc/indexer/idx-external-indexer-20220902T1208/ca.pem", |
"certificate": "/etc/indexer/idx-external-indexer-20220902T1208/worker.pem", |
"enabled": true, |
"private_key": "/etc/indexer/idx-external-indexer-20220902T1208/worker.key" |
} |
}, |
{ |
"address": "192.168.1.118", |
"port": 12345, |
"ssl": { |
"ca_certificate": "/etc/indexer/idx-external-indexer-20220902T1207/ca.pem", |
"certificate": "/etc/indexer/idx-external-indexer-20220902T1207/worker.pem", |
"enabled": true, |
"private_key": "/etc/indexer/idx-external-indexer-20220902T1207/worker.key" |
} |
} |
], |
"settings": { |
"log_level": 3, |
"ocr": { |
"engine": "omnipage-external", |
"minimal_time_distance": 1, |
"ocr_thread_count": 2 |
}, |
"pkcs11": { |
"custom_password": false, |
"slots": [] |
}, |
"terminal": { |
"extract_buffer": true |
}, |
"worker_arguments": "--http-config /opt/external-indexer/httpconfig.json" |
}, |
"worker_groups": [ |
{ |
"balancing": false, |
"capabilities": [ |
"index" |
], |
"count": 4, |
"name": "workers" |
}, |
{ |
"balancing": false, |
"capabilities": [ |
"screenshot", |
"video" |
], |
"count": 1, |
"name": "screenshot-and-video" |
}, |
{ |
"balancing": false, |
"capabilities": [ |
"video" |
], |
"count": 1, |
"name": "video" |
}, |
{ |
"balancing": false, |
"capabilities": [ |
"near-realtime" |
], |
"count": 0, |
"name": "near-realtime" |
}, |
{ |
"balancing": true, |
"capabilities": [], |
"count": 0, |
"name": "shared" |
} |
], |
"workercontroller": { |
"log_level": "info" |
} |
} |
These worker processes disconnect from the service when they finish processing a job, or when they do not receive a job within 60 seconds after connecting to the service.
-
Define additional worker-groups.
You can define any number of worker-groups with different capabilities, but you can have only one dedicated service and one service pool.
-
Add "balancing": true to any worker-group to share those workers between the services configured in the service-pool field.
Worker groups without balancing option, or balancing set to false connect to the dedicated service to fetch jobs.
Uploading decryption keys to the external indexer
If the audit trails you want to index are encrypted, complete the following steps to make the decryption keys available for the indexer.
To make the decryption keys available for the external indexer
-
Obtain the RSA private key and copy it to the external indexer's host.
-
Use the indexer-keys-json utility to transform the private key to the required JSON format. When executed, the script asks for the path to the private key, and the password of the private key. After the conversion, the password is removed.
The utility automatically adds the private key to the /etc/indexer/indexer-keys.cfg keystore file. If you want to use a different keystore file, use the --keystore argument to specify another file. If the keystore already includes the private key you want to add, it will be ignored.
-
In the /opt/external-indexer/usr/bin/ folder, issue the following command: indexer-keys-json
-
Provide the absolute path to the private key. Alternatively, you can include this information as a parameter: indexer-keys-json --private-key <path-to-private-key>
-
If the key is password protected, enter the password to the private key.
-
To add additional keys, re-run the indexer-keys-json command.
-
You can now start the indexer service. For more information, see Starting the external indexer.
Configuring a hardware security module (HSM) or smart card to integrate with external indexer
It is possible to use a hardware security module (HSM) or a smart card to store the decryption keys required for decrypting audit trails. An HSM or a smart card is a tamper-resistant physical, software, or cloud solution that can securely store digital keys used for authentication.
The main steps of configuring a hardware security module (HSM) or smart card to integrate with an external indexer are as follows:
-
Set up and test the environment.
-
Encrypt the PKCS#11 PIN.
To see examples of how to configure various HSM or smart card solutions that you wish to integrate with your external indexer(s), consult the following sections:
Topics:
Setting up and testing the environment
To access an HSM or smart card with the external indexer, a PKCS#11 shared library plugin must be used. In most cases, these libraries also need a background daemon or environment variables set. The PKCS#11 library must be accessible to the external indexer with a proper environment.
To set up the environment and test it, complete the following steps.
-
Load the environment for the indexer commands:
source /etc/indexer/external-indexer.env
-
Test your environment.
-
Option #1: Use the pkcs11-tool to test your environment:
-
List the available slots.
pkcs11-tool --modul <path-to-pkcs11-library> -L
-
List the objects in a slot.
pkcs11-tool --modul <path-to-pkcs11-library> -l --slot <id> -O
-
Option #2: Use the indexerworker with the log level set to dump to see the available keys:
indexerworker -l -v 7 --pkcs11-lib <path-to-pkcs11-library> --pkcs11-slot-id <id> --pkcs11-pin <pin>
-
Assuming that the environment is ready, the external indexer must be configured to use the PKCS#11 library. To do so, edit /etc/indexer/indexerworker.cfg as follows:
...
"settings": {
"pkcs11": {
"custom_password": false
"slots": [
{
"library": "<path-to-pkcs11-library>",
"slot_id": <slot-number>,
"pin": "<your-encrypted-PIN>"
}
]
}
}
...
