This document will guide you through the first steps of using the RPC API. By following these steps you will get a good overview of the capabilities and the basic functionalities of the API.
Prerequisites
Previous experience:
-
You are familiar with syslog-ng Store Box and its search interface
-
You are familiar with related basic concepts, such as an API, REST, HTTP and JSON
-
The examples are provided in the form of UNIX shell commands, we assume that you are comfortable using such an environment
Software requirements:
-
Pre-installed syslog-ng Store Box that has the RPC API feature enabled
-
The "wget" utility is installed. It is widely available for the majority of platforms, and can be installed from http://www.gnu.org/software/wget/.
-
The "jq" utility for command-line JSON formatting and conversion is installed. The source code and binaries for the most common platforms can be downloaded from http://stedolan.github.io/jq/.
Installation
Make sure you have your syslog-ng Store Box installed, configured and running. Throughout this guide we assume that it is accessible at the IP address 1.2.3.4 and that admin/a is a valid username/password combination.
We perform the following operations on the built-in local logspace, so make sure the defaults are not changed and it is configured to receive and index local logs.
The basics
The API to access the logs is a REST-based API that runs over HTTPS. All examples use the wget utility to access it, and we perform HTTP GET queries against SSB. Of course, mature libraries or built-in methods are available for practically all programming environments to interact with a REST-based API providing replies in JSON format, but throughout this guide we use the command line, performing the query with wget and processing the results with a tiny but useful utility called "jq".
A typical query looks like this:
$ wget -q --no-check-certificate --header [auth info, see later] -O - 'https://[IP address of SSB]/api/[API version]/[API command group]/][API command]?[API command arguments] | jq '[filters for the JSON output']
Documentation of the RPC API
The documentation of the SSB RPC API is available online from the following URL: https://<ip-address-of-SSB>/api/4/documentation. This documentation contains the detailed description of public calls, with examples.
Queries going through the API go through the same authentication-authorization-accounting steps as sessions using the UI. There are no special API permissions or users: the sames credentials are used for access over the API. To authenticate yourself, you have to use the login command. It will provide you with an authentication token which is valid until your session times out or you explicitly log off. You have to send back this authentication token in the header of all further requests. In the current beta version this token needs to be sent back as the PHP session ID cookie but this might change in the final version.
SSB prevents brute force attacks when logging in. If you repeatedly try logging in to SSB using incorrect login details within a short period of time (10 times within 60 seconds), the source IP gets blocked on UI destination port 443 for 5 minutes. Your browser displays an Unable to connect page.
-
Let's log in. Remember, we assume that 1.2.3.4 is the IP address of your SSB and admin/a is a valid username/credential pair. Make sure you alter your query according to your environment.
$ SSB_IP=1.2.3.4
$ wget -q --no-check-certificate -O - "https://$SSB_IP/api/1/login" --post-data='username=admin&password=a'
{"result":"af574226cb4d3de28312b27f2bfc705e","error":{"code":null,"message":null},"warnings":[]}
-
OK, this is a bit hard to read, let's have jq pretty-print it:
$ wget -q --no-check-certificate -O - "https://$SSB_IP/api/1/login" --post-data='username=admin&password=a' | jq '.'
{
"warnings": [],
"error": {
"message": null,
"code": null
},
"result": "39d5b0229ff744cdd384d54618d7039c"
}
That's better, and now the structure of responses can be seen. It is always a single JSON object with three parts: warnings, error (these first two are optional and might be omitted when not in used in the final version) and the most important part, the result block. In this case it contains our identifier key. Note that it has changed since the first query: as we logged in again it has been regenerated and the old key is no longer accepted.
-
We need it for later examples, so let's save it into a variable. We use the filtering capabilities of jq to cut out the part we are interested in: the result block itself.
$ SESSID=`wget -q --no-check-certificate -O - "https://$SSB_IP/api/1/login" --post-data='username=admin&password=a' | jq '.result' -r`
$ echo $SESSID
f36a9399c3cbcc3ca5dab037f8374dd8
Now we can move on to perform real queries.
Getting the list of logspaces
Fetching the number of hits
In some cases you might not need the actual results, only their number. To draw charts (as we do in the timeline section of our search UI) or to calculate statistics, usually the number of results is enough. You could always perform a search, fetch all results, and then count their number, but that's not really effective. Instead, you can ask the API to only return the number of hits by replacing filter with number_of_messages in your request and remove the offset/limit settings.
-
To follow up on the previous example, let's see the number of messages containing "starting up" in overall and the number of those sent out by syslog-ng:
$ wget -q --no-check-certificate -O - --header "Cookie: AUTHENTICATION_TOKEN=$SESSID" "https://$SSB_IP/api/1/search/logspace/number_of_messages/local?from=0&to=9999999999&search_expression=starting up" | jq '.result'
84
$ wget -q --no-check-certificate -O - --header "Cookie: AUTHENTICATION_TOKEN=$SESSID" "https://$SSB_IP/api/1/search/logspace/number_of_messages/local?from=0&to=9999999999&search_expression=starting up program:syslog-ng" | jq '.result'
1
