Local services: configuring the indexer
Indexing is a resource intensive (CPU and hard disk) operation, and depending on the number of processed audit trails and parallel connections passing SPS, may affect the performance of SPS. Test it thoroughly before enabling it in a production environment that is under heavy load. If your SPS appliance cannot handle the connections and the indexing, consider using external indexers (see "Configuring external indexers" in the Administration Guide) to decrease the load on SPS. For sizing recommendations, ask your One Identity partner or contact our Support Team.
|
|
NOTE:
Only those audit trails will be processed that were created after full-text indexing had been configured for the connection policy. It is not possible to process already existing audit trails. |
|
|
NOTE:
Using content policies significantly slows down connections (approximately 5 times slower), and can also cause performance problems when using the indexer service. |
URL
GET https://<IP-address-of-SPS>/api/configuration/local_services/indexer
Cookies
| Cookie name | Description | Required | Values |
|---|---|---|---|
| session_id | Contains the authentication token of the user | Required |
The value of the session ID cookie received from the REST server in the authentication response, for example, a1f71d030e657634730b9e887cb59a5e56162860. For details on authentication, see Authenticate to the SPS REST API. Note that this session ID refers to the connection between the REST client and the SPS REST API. It is not related to the sessions that SPS records (and which also have a session ID, but in a different format). |
Sample request
The following command lists the configuration options.
curl --cookie cookies https://<IP-address-of-SPS>/api/configuration/local_services/indexer
Response
The following is a sample response received when external indexers are disabled.
For details of the meta object, see Message format.
{
"body": {
"decryption_keys": [
{
"key": "e38d47bd-5374-4d7c-b683-e26ea77142e2",
"meta": {
"href": "/api/configuration/x509/e38d47bd-5374-4d7c-b683-e26ea77142e2"
}
}
],
"number_of_workers": 1,
"remote_access": {
"enabled": false
},
"selection": "integrated"
},
"key": "indexer",
"meta": {
"first": "/api/configuration/local_services/admin_web",
"href": "/api/configuration/local_services/indexer",
"last": "/api/configuration/local_services/user_web",
"next": "/api/configuration/local_services/postgresql",
"parent": "/api/configuration/local_services",
"previous": "/api/configuration/local_services/admin_web",
"remaining_seconds": 599,
"transaction": "/api/transaction"
}
}
A sample response when external indexers are enabled:
{
"body": {
"decryption_keys": [],
"number_of_workers": 1,
"number_of_workers": 0,
"remote_access": {
"access_restriction": {
"allowed_from": [
"10.40.0.0/16"
],
"enabled": true
},
"enabled": true,
"listen": [
{
"address": {
"key": "nic1.interfaces.ff7574025754b3df1647001.addresses.1",
"meta": {
"href": "/api/configuration/network/nics/nic1#interfaces/ff7574025754b3df1647001/addresses/1"
}
},
"port": 12345
}
],
"ssl_config": {
"ca": {
"key": "52735ce4-4a43-458d-8803-c23c715640a5",
"meta": {
"href": "/api/configuration/x509/52735ce4-4a43-458d-8803-c23c715640a5"
}
},
"service": {
"key": "60eacdba-d889-4cb4-bdb0-cbbd4054f01c",
"meta": {
"href": "/api/configuration/x509/60eacdba-d889-4cb4-bdb0-cbbd4054f01c"
}
},
"worker": {
"key": "93198544-1e82-4661-90b7-e01b0b1e2ed9",
"meta": {
"href": "/api/configuration/x509/93198544-1e82-4661-90b7-e01b0b1e2ed9"
}
}
}
},
"selection": "integrated"
},
"key": "indexer",
"meta": {
"first": "/api/configuration/local_services/admin_web",
"href": "/api/configuration/local_services/indexer",
"last": "/api/configuration/local_services/user_web",
"next": "/api/configuration/local_services/postgresql",
"parent": "/api/configuration/local_services",
"previous": "/api/configuration/local_services/admin_web",
"remaining_seconds": 599,
"transaction": "/api/transaction"
}
}
| Element | Type | Description | ||
|---|---|---|---|---|
| key | string | Top level element, contains the ID of the endpoint. | ||
| body | Top level element (string) | Contains the configuration options of the indexer service. | ||
| decryption_keys | list | Indexing encrypted audit trails requires the X.509 certificates and the matching private keys. The certificates must in PEM format, and use RSA keys. This parameter lists the reference IDs of the configured decryption keys. When configuring the indexer, you must first upload the keys before you can configure the decryption keys. For details, see Private keys stored on SPS. | ||
| key | reference | The ID of the referenced decryption key. You can upload private keys at the /api/configuration/private_key endpoint. For details, see Private keys stored on SPS. | ||
| number_of_near_realtime_workers | integer |
The number of indexer workers configured to perform near-realtime indexing. For details, see "Configuring the external indexer" in the Administration Guide. | ||
| number_of_workers | integer |
This option determines the maximum number of parallel indexing tasks that the SPS appliance performs. The default value is set to the number of detected CPU cores. Note that indexing audit trails requires about 50-100 Mbytes of memory for terminal sessions (SSH, Telnet, TN3270), and 150-300 Mbytes for graphical sessions (RDP, ICA, VNC, X11). Consider the memory usage of your SPS host before modifying this value. | ||
| remote_access | JSON object | Enables external indexers to access the SPS host, and configures access restrictions and other parameters. | ||
| selection | string | The value of this option must be integrated. | ||
| Element | Type | Description | ||||
|---|---|---|---|---|---|---|
|
access_restriction |
JSON object |
Enables and configures limitations on the clients that can access the web interface, based on the IP address of the clients. | ||||
|
allowed_from |
list |
The list of IP networks from where the administrators are permitted to access this management interface. To specify the IP addresses or networks, use the IPv4-Address/prefix format, for example, 10.40.0.0/16. | ||||
| enabled |
boolean |
Set it to true to restrict access to the specified client addresses. | ||||
| enabled | boolean |
Enables the remote access for the external indexers. That way, indexer services running on external hosts can access the audit trails, index them, and upload the indexed data to SPS. If this option is set to False, SPS ignores every other option of this object. For details on installing and configuring external indexers, see "Configuring external indexers" in the Administration Guide.
| ||||
| listen | list | Selects the network interface, IP address, and port where the clients can access the web interface. | ||||
| address | JSON object |
A reference to a configured network interface and IP address where this local service accepts connections. For example, if querying the interface /api/configuration/network/nics/nic1#interfaces/ff7574025754b3df1647001/addresses/ returns the following response: {
"body": {
"interfaces": {
"@order": [
"ff7574025754b3df1647001"
],
"ff7574025754b3df1647001": {
"addresses": {
"1": "10.40.255.171/24",
"@order": [
"1"
]
},
"name": "default",
"vlantag": 0
}
},
"name": "eth0",
"speed": "auto"
},
"key": "nic1",
"meta": {
"first": "/api/configuration/network/nics/nic1",
"href": "/api/configuration/network/nics/nic1",
"last": "/api/configuration/network/nics/nic3",
"next": "/api/configuration/network/nics/nic2",
"parent": "/api/configuration/network/nics",
"previous": null,
"transaction": "/api/transaction"
}
}
Then the listening address of the local service is the following. nic1.interfaces.ff7574025754b3df1647001.addresses.1 This is the format you have to use when configuring the address of the local service using REST: "address": "nic1.interfaces.ff7574025754b3df1647001.addresses.1" When querying a local services endpoint, the response will contain a reference to the IP address of the interface in the following format: "address": {
"key": "nic1.interfaces.ff7574025754b3df1647001.addresses.1",
"meta": {
"href": "/api/configuration/network/nics/nic1#interfaces/ff7574025754b3df1647001/addresses/1"
}
}, | ||||
| port | integer |
The port number where this local service accepts connections. | ||||
| ssl_config | JSON object | Contains references to the certificates used to encrypt the communication between SPS and the external indexer hosts. SPS generates these certificates automatically when you enable the indexer service. | ||||
| ca | reference | The ID of the CA certificate used to sign the certificates used to communicate between SPS and the external indexers. | ||||
| service | reference | The ID of the certificate that SPS shows to the external indexer hosts. | ||||
| worker | reference | The ID of the certificate that the external indexer hosts must show to SPS. | ||||
Status and error codes
The following table lists the typical status and error codes for this request. For a complete list of error codes, see Application level error codes.
| Code | Description | Notes |
|---|---|---|
| 401 | Unauthenticated | The requested resource cannot be retrieved because the client is not authenticated and the resource requires authorization to access it. The details section contains the path that was attempted to be accessed, but could not be retrieved. |
| 401 | AuthenticationFailure | Authenticating the user with the given credentials has failed. |
| 404 | NotFound | The requested object does not exist. |
Updating the indexer configuration
To update the configuration of the indexer, you have to PUT the updated configuration in JSON format to the endpoint, for example:
{
"decryption_keys": ["216b33dd-a1cd-41b1-85c5-66290b7a043d"],
"number_of_near_realtime_workers": 0,
"number_of_workers": 2,
"remote_access": {
"access_restriction": {
"allowed_from": [
"10.40.0.0/16"
],
"enabled": true
},
"enabled": true,
"listen": [
{
"address": "nic1.interfaces.ff7574025754b3df1647001.addresses.1",
"port": 12354
}
],
"ssl_config": {
"ca": "773ed50d-3066-44f1-84ec-cbef59111702",
"service": "a8b6c791-c24a-466d-ac50-a425a5253d46",
"worker": "c54c436f-63c5-4a2e-a59e-7ad904bbf0f2"
}
},
"selection": "integrated"
}
Indexer policies
Indexer policies allow you to configure the Optical Character Recognition (OCR) engine of SPS, and specify which languages it should use. Only graphical protocols (RDP, Citrix ICA, VNC) are affected.
If you specify the languages manually, note the following:
-
Specifying only one language provides the best results in terms of performance and precision.
-
The English language is always detected along with the non-English languages that you have configured. However, if you want the OCR to only recognize the English language, you have to select it from the list of languages.
-
There are certain limitations in the OCR engine when recognizing languages with very different character sets. For this reason, consider the following:
-
When selecting Asian languages (Simplified Chinese, Traditional Chinese, Korean), avoid adding languages that use the Latin alphabet.
-
When selecting the Arabic language, avoid selecting any other languages.
-
The Thai language is currently not supported. If you are interested in using SPS to index Thai texts, contact our Sales Team.
-
URL
GET https://<IP-address-of-SPS>/api/configuration/policies/indexing
Cookies
| Cookie name | Description | Required | Values |
|---|---|---|---|
| session_id | Contains the authentication token of the user | Required |
The value of the session ID cookie received from the REST server in the authentication response, for example, a1f71d030e657634730b9e887cb59a5e56162860. For details on authentication, see Authenticate to the SPS REST API. Note that this session ID refers to the connection between the REST client and the SPS REST API. It is not related to the sessions that SPS records (and which also have a session ID, but in a different format). |
Sample request
The following command lists the available indexer policies.
curl --cookie cookies https://<IP-address-of-SPS>/api/configuration/policies/indexing
The following command displays a specific indexer policy.
curl --cookie cookies https://<IP-address-of-SPS>/api/configuration/policies/indexing/<id-of-the-policy>
Response
The following is a sample response received when querying the /api/configuration/policies/indexing/ endpoint.
For details of the meta object, see Message format.
{
"items": [
{
"key": "-50000",
"meta": {
"href": "/api/configuration/policies/indexing/-50000"
}
},
{
"key": "13442970955825a89b55e46",
"meta": {
"href": "/api/configuration/policies/indexing/13442970955825a89b55e46"
}
}
],
"meta": {
"first": "/api/configuration/policies/audit_policies",
"href": "/api/configuration/policies/indexing",
"last": "/api/configuration/policies/usermapping_policies",
"next": "/api/configuration/policies/ldap_servers",
"parent": "/api/configuration/policies",
"previous": "/api/configuration/policies/credentialstores",
"remaining_seconds": 599,
"transaction": "/api/transaction"
}
}
A sample response when querying a specific indexer policy:
{
"body": {
"index": {
"command": true,
"keyboard": false,
"mouse": false,
"screen_content": false,
"window_title": true
},
"name": "english-german-russian",
"ocr": {
"accuracy": "accurate"
"custom_languages": {
"enabled": true,
"languages": [
"eng",
"deu",
"rus"
]
}
}
},
"key": "-50000",
"meta": {
"first": "/api/configuration/policies/indexing/-50000",
"href": "/api/configuration/policies/indexing/-50000",
"last": "/api/configuration/policies/indexing/-50000",
"next": null,
"parent": "/api/configuration/policies/indexing",
"previous": null,
"remaining_seconds": 599,
"transaction": "/api/transaction"
}
}
| Element | Type | Description | ||
|---|---|---|---|---|
| key | string | Top level element, contains the ID of the policy. | ||
| body | Top level element (string) | Contains the configuration options of the indexer policy. | ||
| index | Top level element |
Contains the indexed events of the indexer policy. Possible values:
| ||
| name | string | The name of the indexer policy. | ||
| ocr | JSON object | Configuration of the OCR engine. | ||
| accuracy | string |
Accuracy level for Optical Character Recognition. Possible values:
| ||
| custom_languages | Top level element | Configures what languages to detect. | ||
| Custom languages elements | Type | Description | |
|---|---|---|---|
| custom_languages | Top level element | Configures what languages to detect. | |
| enabled | boolean | If false, the OCR engine detects the language of the text automatically. This is the default behavior. To specify which languages to use, set the custom_languages element to true, and list the abbreviation of the languages in the languages element (for example, "eng", "ger"). | |
| languages | list |
The list of languages the OCR engine should use to process graphical protocols. To specify which languages to use, set the custom_languages element to true, and list the abbreviation of the languages in the languages element (for example, "eng", "ger").
The following languages are supported: English: eng, German: deu, French: fra, Dutch: nld, Norwegian: nor, Swedish: swe, Finnish: fin, Danish: dan, Icelandic: isl, Portuguese: por, Spanish: spa, Catalan: cat, Galician: glg, Italian: ita, Maltese: mlt, Greek: ell, Polish: pol, Czech: ces, Slovak: slk, Hungarian: hun, Slovenian: slv, Croatian: hrv, Romanian: ron, Albanian: sqi, Turkish: tur, Estonian: est, Latvian: lav, Lithuanian: lit, Esperanto: epo, Serbian(Latin): qsl, Serbian: srp, Macedonian: mkd, Moldavian: mol, Bulgarian: bul, Byelorussian: bel, Ukrainian: ukr, Russian: rus, Chechen: che, Kabardian: kbd, Afrikaans: afr, Aymara: aym, Basque: eus, Bemba: bem, Blackfoot: bla, Breton: bre, Brazilian: qbp, Bugotu: bgt, Chamorro: cha, Tswana(Chuana): tsn, Corsican: cos, Crow: cro, Eskimo: qes, Faroese: fao, Fijian: fij, Frisian: fry, Friulian: fur, Gaelic(Irish): gle, Gaelic(Scottish): gla, Ganda(Luganda): lug, Guarani: grn, Hani: hni, Hawaiian: haw, Ido: ido, Indonesian: ind, Interlingua: ina, Kasub: csb, Kawa: wbm, Kikuyu: kik, Kongo: kon, Kpelle: kpe, Kurdish: kur, Latin: lat, Luba: lua, Luxembourgish: ltz, Malagasy: mlg, Malay: msa, Malinke: mlq, Maori: mri, Mayan: MYN, Miao: hmn, Minangkabau: min, Mohawk: moh, Nahuatl: NAH, Nyanja: nya, Occidental: ile, Ojibway: oji, Papiamento: pap, PidginEnglish: tpi, Provencal: oci, Quechua: que, Rhaetic: roh, Romany: rom, Rwanda: kin, Rundi: run, Samoan: smo, Sardinian: srd, Shona: sna, Sioux: dak, Sami: SMI, Sami(Lule): smj, Sami(Northern): sme, Sami(Southern): sma, Somali: som, Sotho: sot, Sundanese: sun, Swahili: swa, Swazi: ssw, Tagalog: tgl, Tahitian: tah, Tinpo: qti, Tongan: ton, Tun: tug, Visayan: qis, Welsh: cym, Sorbian(Wend): WEN, Wolof: wol, Xhosa: xho, Zapotec: zap, Zulu: zul. | |
Status and error codes
The following table lists the typical status and error codes for this request. For a complete list of error codes, see Application level error codes.
| Code | Description | Notes |
|---|---|---|
| 201 | Created | The new resource was successfully created. |
| 401 | Unauthenticated | The requested resource cannot be retrieved because the client is not authenticated and the resource requires authorization to access it. The details section contains the path that was attempted to be accessed, but could not be retrieved. |
| 401 | AuthenticationFailure | Authenticating the user with the given credentials has failed. |
| 404 | NotFound | The requested object does not exist. |
| 409 | Conflict | No open Transaction is available. Open a transaction before using this request. For details, see Open a transaction. |
Add an indexing policy
To add an indexing policy, you have to:
-
Open a transaction.
For details, see Open a transaction.
-
Create the JSON object for the new indexing policy.
You can find a detailed description of the available parameters listed in Element .
POST the JSON object to the https://<IP-address-of-SPS>/api/configuration/policies/indexing endpoint. If the POST request is successful, the response includes the key of the new ticketing policy. For example:
{ "key": "aa423b72-0d0f-4275-be30-494e9a99ffad", "meta": { "href": "/api/configuration/policies/indexing/aa423b72-0d0f-4275-be30-494e9a99ffad", "parent": "/api/configuration/policies/indexing", "transaction": "/api/transaction" } } -
Commit your changes.
For details, see Commit a transaction.
Reporting
Topics:
Reporting
List of endpoints for configuring reporting, and accessing the generated reports.
URL
GET https://<IP-address-of-SPS>/api/configuration/reporting
Cookies
| Cookie name | Description | Required | Values |
|---|---|---|---|
| session_id | Contains the authentication token of the user | Required |
The value of the session ID cookie received from the REST server in the authentication response, for example, a1f71d030e657634730b9e887cb59a5e56162860. For details on authentication, see Authenticate to the SPS REST API. Note that this session ID refers to the connection between the REST client and the SPS REST API. It is not related to the sessions that SPS records (and which also have a session ID, but in a different format). |
Sample request
The following command lists the available endpoints.
curl --cookie cookies https://<IP-address-of-SPS>/api/configuration/reporting
Response
The following is a sample response received when listing the available endpoints.
For details of the meta object, see Message format.
{
"meta": {
"first": "/api/configuration/aaa",
"href": "/api/configuration/reporting",
"last": "/api/configuration/x509",
"next": "/api/configuration/ssh",
"parent": "/api/configuration",
"previous": "/api/configuration/rdp",
"transaction": "/api/transaction"
},
"items": [
{
"key": "content_subchapters",
"meta": {
"href": "/api/configuration/reporting/content_subchapters"
}
},
{
"key": "custom_subchapters",
"meta": {
"href": "/api/configuration/reporting/custom_subchapters"
}
},
{
"key": "predefined_reports",
"meta": {
"href": "/api/configuration/reporting/predefined_reports"
}
},
{
"key": "reports",
"meta": {
"href": "/api/configuration/reporting/reports"
}
},
{
"key": "statistics_subchapters",
"meta": {
"href": "/api/configuration/reporting/statistics_subchapters"
}
}
]
}
| Endpoint | Description |
|---|---|
| content_subchapters | List of the reporting subchapters created from audit trail content (statistics of search keywords, and screenshots). |
| custom_subchapters | List of the reporting subchapters created from custom queries to the SPS connection database. |
| predefined_reports | List of the pre-defined reports available on SPS. |
| reports | List of the configured reports. |
| statistics_subchapters | List of the reporting subchapters created from connection statistics. |
Status and error codes
The following table lists the typical status and error codes for this request. For a complete list of error codes, see Application level error codes.
| Code | Description | Notes |
|---|---|---|
| 201 | Created | The new resource was successfully created. |
| 400 | InvalidQuery | The requested filter or its value is invalid. |
| 401 | Unauthenticated | The requested resource cannot be retrieved because the client is not authenticated and the resource requires authorization to access it. The details section contains the path that was attempted to be accessed, but could not be retrieved. |
| 401 | AuthenticationFailure | Authenticating the user with the given credentials has failed. |
| 404 | NotFound | The requested object does not exist. |