Chapter 11. Tungsten REST API (APIv2)

Version 7.0.0 of the Tungsten suite of products introduced the new Public RESTful API (Referred to as APIv2)

APIv2 will allow quick and easy access to monitoring, cluster manipulation and change of configuration for both humans and software.

A number of key development decisions where made to allow the release of the first version of the new API. The choice was made to release with a high level of security by default, and a minimal set of features so we can release quickly. Future releases will build on this initial foundation:

  • Security First: the API is only enabled on localhost (127.0.0.1) by default. Additionally, SSL is now enabled by default (in previous releases, SSL was disabled by default).

  • User/Password Protection: HTTP Basic Auth is required to access most API calls. RBAC will come in a future version.

  • Per-layer API: each component has its own API.

  • Per-host instances: Passwords are stored securely on each host. Of course, the Tungsten installation tool (tpm) allows deployment of identical user/password pairs across nodes, as does the Tungsten Dashboard GUI.

  • The REST API provides read-only information through HTTP GET calls. Continuent has made the deliberate choice of using only POST actions when they affect the state or configuration of the cluster. The reason why we made that choice is a matter of simplification: there is an overlap between the definitions of PUT and POST, and the decision to chose between the two is often a balance with pros and cons, triggering sterile discussions. Rather that spending time debating and choosing, we decided to offer only one, so you will not find any use of PUT in our list of functions.

For the full developer documentation, listing all available calls, please refer to the following links:

Each layer of the product (Manager, Replicator, Proxy) has its own API, allowing for granular access and control of all cluster operations.

Warning

API calls triggering configuration changes are protected by a flag, i-am-sure=true, in order to avoid unwanted, potentially dramatic, configuration changes. This applies to:

  • configuration/module/servicesmap

  • reset

  • offline

  • online

  • onhold

  • addDataService

  • addDataSource

11.1. Getting Started with Tungsten REST API

11.1.1. Configuring the API

11.1.1.1. Network Ports

Each cluster process listens for API calls using the following ports by default:

Port

Component

8090

Manager

8096

Connector (Proxy)

8097

Replicator

Should you wish to choose your own ports for the API, this can be handled by specifying the new ports in the tpm config using the following properties:

connector-rest-api-port=8096
manager-rest-api-port=8090
replicator-rest-api-port=8097

11.1.1.2. User Management

With our focus on security in the first release of the API, without any user created, only ping and createAdminUser calls will be available. tpm makes it easy to create the administration user at install through the following options:

rest-api-admin-user=tungsten
rest-api-admin-password=secret

Once an admin user is created, all other APIs call will be available using basic HTTP authentication with that user/password.

If you haven't created the user as part of the initial cluster installation, you can use the Dashboard GUI to do this for you during Dashboard configuration, or use the CLI calls explained below.

Note

If you plan to use the Dashboard GUI, you MUST ensure the API User has been created to alow the Dashboard to function correctly.

Using CURL commands

The createAdminUser call will allow creation and modification of REST API users.

shell> curl -k -H 'Content-type: application/json' --request POST 'https://127.0.0.1:8096/api/v2/createAdminUser?i-am-sure=true' \
> --data-raw '{
>   "payloadType": "credentials",
>   "user":"tungsten",
>   "pass":"security"
> }'

{ "payloadType":"StringPayload",
  "payloadVersion":"1",
  "payload":{ 
      "value":"https://127.0.0.1:8096/api/v2/user/tungsten"
            }
}

Using tapi

The tapi tool can make the creation of the admin user simple and straightforward, the following example shows the use of this approach.

shell> tapi --create --host hostnamehere --create-user usernamehere --create-password passwordhere -v

Important

Note that users created/modified through the curl and tapi calls only apply to the host on which the call was done. The same, identical call, will have to be re-run on each host of the cluster.

In order to make that last requirement clear, the URL parameter ?i-am-sure=true will need to be passed to createAdminUser when issued via the curl command as per the example above.

Important

Usernames must NOT contain any of the following reserved characters: (space) ! # $ & ' ( ) * + , / : ; = ? @ [ ]

11.1.1.3. SSL/Encryption

SSL is enabled by default and will use the same keystore, certificates and aliases as the cluster TLS ones.

You can specify them with java-tls-keystore-path and java-truststore-path tpm options, or let tpm generate the certificates for you during installation.

For more detailed information on SSL, see Chapter 5, Deployment: Security

11.1.1.4. Enabling and Disabling the API

As previously mentioned, the API is enabled by default, listening only on localhost.

To disable the API, you will need to specify the following tpm options:

replicator-rest-api=false 
manager-rest-api=false 
connector-rest-api=false

To change the address that the API is listening on, you need to specify the following tpm options:

replicator-rest-api-address=0.0.0.0
connector-rest-api-address=0.0.0.0
manager-rest-api-address=0.0.0.0

11.1.2. How to Access the API

11.1.2.1. CURL calls and Examples

curl is the simplest command line tool to access the API.

Note that in the examples below, we use self signed certificates, which is why the -k flag is passed to cURL

Ping command

shell> curl -k --request GET 'https://127.0.0.1:8096/api/v2/ping'

{ "payloadType":"PingPayload",
  "payloadVersion":"1",
  "payload":{
      "message":"Ping test",
      "date":"Mon Sep 20 11:48:48 CEST 2021",
      "hostName":"gilmbp-8.local",
      "pid":16743,
      "jvmUptime":152513
      }
}

In this example, we sent a simple ping. Result comes as an HTTP OK response with a payload containing a string message “ping test”, the date of execution, the hostname of the executor and its pid, plus the number of milliseconds elapsed since the java executable was started

Create admin user

shell> curl -k -H 'Content-type: application/json' --request POST 'https://127.0.0.1:8096/api/v2/createAdminUser?i-am-sure=true' \
> --data-raw '{
>   "payloadType": "credentials",
>   "user":"tungsten",
>   "pass":"security"
> }'

{ "payloadType":"StringPayload",
  "payloadVersion":"1",
  "payload":{ 
      "value":"https://127.0.0.1:8096/api/v2/user/tungsten"
            }
}

This POST request show how input data can be passed to API calls as a json string.

In return, the createAdminUser function will give a link to the API call allowing you to display details for the user we just created. Let’s follow this link:

shell> curl -k --request GET 'https://127.0.0.1:8096/api/v2/user/tungsten' -utungsten:security

{ "payloadType":"CredentialsPayload",
  "payloadVersion":"1",
  "payload":{
      "user":"tungsten",
      "pass":"**obfuscated**",
      "access":"full"
            }
}

This call shows how the user/password tuple is passed.

The equivalent call using base64 encoded Authorization header, obtained with echo -ne "tungsten:security" | base64 would look something like the following:

shell> curl -k --request GET 'https://127.0.0.1:8096/api/v2/user/tungsten' --header 'Authorization: Basic dHVuZ3N0ZW46c2VjdXJpdHk='

  "payload":{
      "user":"tungsten",
      "pass":"**obfuscated**",
      "access":"full"
            }
}

11.1.2.2. tapi

tapi is a convenient command line tool developed by Continuent that will ease access to Tungsten components APIs without having to remember full REST call URLs.

Full documentation on tapi can be found here: Section 9.22, “The tapi Command”

11.1.2.3. External Tools

PostMan is one of the most popular GUI tools for REST API testing and use

11.1.3. Data Structures

11.1.3.1. Generic Payloads

Tungsten API defines its own payloads for both inputs and output. The generic structure looks like the following:

{
   "payloadType": "TypeOfPayload",
   "payloadVersion": "1",
   "payload": 
   {
     "key"="value"
   }
}

Where payloadType announces the type of data that will be contained in payload in the given payloadVersion

As an example, a very simple payload is found in the StringPayload data structure and only consists in a key/value pair:

{
   "payloadType": "StringPayload",
   "payloadVersion": "1",
   "payload": 
   {
      "string"="stringvalue"
   }
}

11.1.3.2. INPUT and OUTPUT payloads

When starting up with a fresh installation of tungsten, if no admin user has been provided to tpm, credentials can be sent to the various functions via the following payload

{
    "payloadType": "CredentialsPayload",
    "payload":
    {
     "user":"<user>",
     "pass":"<password>"
    }
}

The same payload structure, slightly enriched, will be found in response to listing the user via:

{
    "payloadType": "CredentialsPayload",
    "payloadVersion": "1",
    "payload": {
        "user": "tungsten",
        "pass": "<obfuscated>",
        "access": "full"
    }
}

Various other payloads used and produced by Tungsten REST API entry point will be found in the detailed technical documentation. Links below:

11.1.3.3. TAPI Datastructures