You can share your worker resources between multiple indexer services, for example behind a load-balancer, or in an SPS cluster if you want to share some of your indexer workers between multiple SPS instances.
To enable resource sharing
-
Use the indexer-box-config script with an additional --service-pool option to enumerate the service configurations.
NOTE: Running this script can overwrite your custom modifications in your indexerworker.cfg file. Make a backup of indexerworker.cfg before running the script.
indexer-box-config
cp /etc/indxer/indexerworker.cfg /etc/indxer/indexerworker.cfg.bak
indexer-box-config $dedicated-server-config --service_pool $cfg1 $cfg2 $cfg3
If you run the script with --service-pool option, the script creates a service-pool field in your configuration file which can be used by any worker-group. The script also creates a default worker-group called shared, with an additional balancing field. These workers run in one-shot mode, and connect to an indexer-service randomly selected from the pool.
Example: output after running script with two configuration files
{
"service": {
"address": "169.254.1.1",
"port": 12345,
"ssl": {
"ca_certificate": "/etc/indexer/cacert.pem",
"certificate": "/etc/indexer/worker.pem",
"enabled": false,
"private_key": "/etc/indexer/worker.key"
}
},
"service_pool": [
{
"address": "192.168.1.111",
"port": 12345,
"ssl": {
"ca_certificate": "/etc/indexer/idx-external-indexer-20220902T1208/ca.pem",
"certificate": "/etc/indexer/idx-external-indexer-20220902T1208/worker.pem",
"enabled": true,
"private_key": "/etc/indexer/idx-external-indexer-20220902T1208/worker.key"
}
},
{
"address": "192.168.1.118",
"port": 12345,
"ssl": {
"ca_certificate": "/etc/indexer/idx-external-indexer-20220902T1207/ca.pem",
"certificate": "/etc/indexer/idx-external-indexer-20220902T1207/worker.pem",
"enabled": true,
"private_key": "/etc/indexer/idx-external-indexer-20220902T1207/worker.key"
}
}
],
"settings": {
"log_level": 3,
"ocr": {
"engine": "omnipage-external",
"minimal_time_distance": 1,
"ocr_thread_count": 2
},
"pkcs11": {
"custom_password": false,
"slots": []
},
"terminal": {
"extract_buffer": true
},
"worker_arguments": "--http-config /opt/external-indexer/httpconfig.json"
},
"worker_groups": [
{
"balancing": false,
"capabilities": [
"index"
],
"count": 4,
"name": "workers"
},
{
"balancing": false,
"capabilities": [
"screenshot",
"video"
],
"count": 1,
"name": "screenshot-and-video"
},
{
"balancing": false,
"capabilities": [
"video"
],
"count": 1,
"name": "video"
},
{
"balancing": false,
"capabilities": [
"near-realtime"
],
"count": 0,
"name": "near-realtime"
},
{
"balancing": true,
"capabilities": [],
"count": 0,
"name": "shared"
}
],
"workercontroller": {
"log_level": "info"
}
}
These worker processes disconnect from the service when they finish processing a job, or when they do not receive a job within 60 seconds after connecting to the service.
-
Define additional worker-groups.
You can define any number of worker-groups with different capabilities, but you can have only one dedicated service and one service pool.
-
Add "balancing": true to any worker-group to share those workers between the services configured in the service-pool field.
Worker groups without balancing option, or balancing set to false connect to the dedicated service to fetch jobs.