The following prerequisites are common across both deployment methods (docker-compose and helm):

In order to install Tungsten Dashboard using the Docker Compose plugin you will require a host configured with the following minimum requirements:

shell> docker --version

In order to install Tungsten Dashboard in Kubernetes you will require a staging host to initiate the installation, with a minimum of the following requirements:

Additionally your cluster must have an access to the docker image file provided in the installation package. Our recommendation is for you to host a private registry of your own in order to manage the images in a centralized manner. This is accepted as the best practice for Kubernetes.

For steps on uploading the images to a Container Registry, see Section 2.3.4, “Upload tungsten-dashboard to Container Registry”

Alternatively you can upload the images as local images directly to the cluster you are using and rely on the local registry of the cluster, please bear in mind how you achieve this depends on the details of your cluster. In addition you should alter the image pull policies in the provided yaml files to match the cluster configuration.

shell> kubectl version --client

shell> helm version

shell> aws eks --region <region-code> update-kubeconfig --name <cluster-name>

shell> docker load -i images/tungsten-dashboard_{{ version_postfix }}-linux_{{ arch }}.tar

shell> aws ecr get-login-password --region <region> | docker login --username AWS --password-stdin <registry ID>

shell> docker tag tungsten-dashboard:{{ version_postfix }} {{ registry ID }}/tungsten-dashboard:{{ version_postfix }}
shell> docker push {{ registry ID }}/tungsten-dashboard:{{ version_postfix }}

Download the Tungsten Dashboard TAR package from the Continuent Download Portal and place the file into the /opt/continuent/software directory on the staging host. From there, unpack the file:

shell> cd /opt/continuent/software
shell> tar zxvf tungsten-dashboard-8.0.4-67.tar.gz
shell> cd /opt/continuent/software/tungsten-dashboard-8.0.4-67

The install.pl command will update an existing configuration if it is executed from within the same directory that was previously used for the initial installation.

The update will detect modifications to the various configuration variables and during the process you will be prompted for the same configuration options as when the installation was performed.

This process could be thought of as the Tungsten Clustering equivalent of tpm update

The install.pl command will upgrade an existing configuration if it is executed from within the new directory which was created by unpacking the software archive..

The upgrade will prompt for the same configuration options as when the initial installation was performed, and detect an existing Tungsten Dashboard version running. You will then be prompted to continue. An example of the process detecting the running dashboard is as follows:

Example of a detected Docker deployment:

Docker service 'tungsten-dashboard' is already running.
CONTAINER ID   IMAGE                           COMMAND                  CREATED              STATUS                        PORTS     NAMES
52594391c843   tungsten-dashboard:8.0.0        "java -jar /app/dash…"   40 minutes ago       Up 40 minutes (healthy)                 tungsten-dashboard-service-1

Do you want to upgrade the existing deployment? [y/N]:

Example of a detected kubernetes deployment:

Kubernetes deployment 'tungsten-dashboard' already exists.
NAME                 READY   UP-TO-DATE   AVAILABLE   AGE
tungsten-dashboard   1/1     1            1           40m14s

Do you want to upgrade the existing deployment? [y/N]:

This process could be thought of as the Tungsten Clustering equivalent of tools/tpm update --replace-release

Once logged in, you are presented with the following view:

Figure 5.1. Dashboard Front Page Overview

An explanation of each highlighted area is as follows:

Navigation

On the left you can see the primary navigation. Hover over it to expand it, and use the button at the bottom of the side bar to lock it in place.

Cluster Filters

At the top you have the cluster filters. Clicking any of them will take you to the Clusters page and show you the clusters that are reporting the filtered status.

Search & Notifications

On the far right of the top bar you have:

To start using the Dashboard the first recommended action is to create your own user account. Login with the installation default account, then open the sidebar by hovering over it and click "User Management". On this page click the "Add" button and fill in the details of the new account. Next, click the logout button and log back in using the account you just created.

In Summary:

Figure 5.2. Add User Form

User accounts are stored in a file on the Dashboard server. This file is intended to be stored in the persistent volume of the Dashboard. In that file you'll find the email, username and hash of the password. Future logins work by matching the incoming hash against the stored one.

Each login returns a JWT token that the browser uses to communicate with the backend. This signed token contains basic JWT details and the username. While the content of the token can be decoded with base64, altering the token in any way will cause the backend to reject it. When a token is rejected, the backend closes the websocket session from the browser.

Once you've logged in, click the "Add cluster" button on the fron tpage (see Figure 5.3, “Cluster Front Page - Add Cluster”) or the clusters page. This opens up a form which asks for the following details:

Figure 5.3. Cluster Front Page - Add Cluster

Figure 5.4. Add Cluster Form

When cluster connection details are added to the Dashboard, the changes are written to both the clusters.json file and into the Dashboard's active memory. The Dashboard's Topology sub-system is aware when these changes occur, and will gather the new details from the cluster via the RestAPI, then store the response into the /app/persistent/dynamic/clusters.edn file, which keeps a map of the hosts by cluster.

When the Dashboard connects to a cluster, one host is selected for each service in the clusters.edn file, and that host is then called via the Rest API to get the initial data for the cluster. This is followed by a TCP connection to the cluster which updates the data in real time.

Each cluster connection is hosted in a virtual thread inside the JVM. If an error occurs, the Dashboard attempts to select a new host and restart the cluster connection unless the error is significant enough (i.e. "invalid certification" or credential data), at which point the Dashboard stops further connection attempts.

Once you've added the desired clusters, you can use the Dashboard or Clusters pages to monitor the status of these clusters in real-time. While the Dashboard page provides limited data, the Clusters page allows you to view detailed information and execute corrective actions to a cluster as needed.

From this page you can find a quick status overview of any of the clusters you have connected the Dashboard to. Clusters are split into small "widgets" on the page. (See Figure 6.1, “Front Page Interface”)

Each widget shows:

Figure 6.1. Front Page Interface

On the Clusters page you can review your cluster status details, execute cluster actions and add or remove clusters from the Dashboard. (See Figure 6.2, “Populated Cluster Front Page”)

Figure 6.2. Populated Cluster Front Page

The first tools available to you on the Clusters page are the Collapse/Expand and Simple/Advanced View toggle buttons just below the top navigation bar. (See Figure 6.3, “Cluster Tool Bar”)

Collapse/Expand opens or closes all clusters visible on the page, and the Simple/Advanced View toggle button switches which columns are shown to you.

Figure 6.3. Cluster Tool Bar

Below the toolbar you can find the clusters themselves:

Figure 6.4. Cluster Header Detail

A description of each item is as follows:

Figure 6.5. Parent Service Actions/Operations

Following the cluster service actions are the Dashboard-specific actions for each service:

Figure 6.6. Edit Cluster Form

The cluster's host details are displayed as a table. (See Figure 6.7, “Cluster Host Table (CAA)”) It has fields such as:

Figure 6.7. Cluster Host Table (CAA)

In the host row actions dropdown (See Figure 6.8, “Primary Host Operations” & Figure 6.9, “Replica Host Operations”) you can find a set of operations the Dashboard can post to the cluster's REST API for each of the hosts. Note that the host's Role affects what actions are visible. To see more details about operations see Chapter 9, Operations

Figure 6.8. Primary Host Operations

Figure 6.9. Replica Host Operations

This page lists the current users stored in the Dashboard with their role, username and email. (See Figure 6.10, “User Management Page”)

From here you can add or remove users as you desire. (See Figure 6.11, “Add User Form”)

Figure 6.10. User Management Page

Figure 6.11. Add User Form

The install.pl script is a command-line tool for installing, updating and upgrading the Tungsten Dashboard. This document provides a detailed guide on how to use this script.

Usage

Where:

For example, to install using Docker, you would run:

shell> ./install.pl -m docker-compose

To install using Kubernetes, you would run:

shell> ./install.pl -m kubernetes

If no method is specified, the script will prompt you to choose a method interactively.

Prerequisites check

The script checks for prerequisites based on the selected installation method:

If the required tools are not found, the script will display an error message and exit. For more information on prerequisites, see Chapter 2, Prerequisites

Interactive Prompts

During execution, the script will prompt you for various pieces of information, such as the admin username and password, the domain for your application, and the path for your application. Please follow the prompts and provide the necessary information.

For details on Updating and Upgrading an existing installation, see Chapter 4, Updates & Upgrades

getcert.pl is a script designed to fetch certificates from a specified remote host. Once fetched, these certificates can be copied and pasted into the dashboard form, or included in the values.yaml during installation.

Usage

./getcert.pl [--api|--tcp|--help] [--port PORT] [--base64] {HOSTNAME|HOSTNAME:PORT}

Where:

Examples

Fetch the API certificate from a host:

shell> ./getcert.pl --api db1.continuent.com

Fetching API certificate from db1.continuent.com:8090 ...

-----BEGIN CERTIFICATE-----
MIIDBDCCAeygAwIBAgIBAzANBgkqhkiG9w0BAQsFADA8MTowOAYDVQQDDDFNeVNR
... rest of certificate content ...
LwZn40KuWnk=
-----END CERTIFICATE-----

Please copy this API certificate (including BEGIN and END markers) into the UI form

Fetch the TCP certificate from a host on a specific port:

shell> ./getcert.pl --tcp --port 11999 {HOSTNAME}

Redirecting Output to a File

The output of the script can be redirected to a file using the > operator. For example:

shell> ./getcert.pl --api {HOSTNAME} > cert.txt

This will save the certificate content into cert.txt without any additional verbose output.

Fetch base64 encoded API certificate from a host for a kubernetes values.yaml file:

shell> ./getcert.pl --api db1.continuent.com --base64

Fetching API certificate from db1.continuent.com:8090 ...

LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURKekNDQWcrZ0F3SUJBZ0l
FS0VUclZUQU5CZ2txaGtpRzl3MEJBUXNGQURCRU1Rc3dDUVlEVlFRR0V3SlYKVX
... rest of base64 certificate content ...

Please copy this API certificate encoded in base64 into the values.yaml configuration file

letsencrypt2jetty.pl is a script that converts a Let's Encrypt-provided certificate into a Dashboard-ready JKS file.

To use this tool you will need to ensure you have the fullchain.pem and privkey.pem files from Let's Encrypt available.

Usage

Where:

Below is an example of converting an existing cert issued by Let's Encrypt using our provided tool letsencrypt2dashboard.pl, which calls the openssl and Java keytool commands:

shell> sudo ./letsencrypt2jetty.pl -d dashdev.continuent.com

Keystore file's password: tungsten
Creating `./cert/`
>>> ACTION: Converting the Let's Encrypt source files to P12 format:
SOURCE:
  /etc/letsencrypt/live/dashdev.continuent.com/fullchain.pem
  /etc/letsencrypt/live/dashdev.continuent.com/privkey.pem
TARGET
  cert/jetty.p12

SUCCESS: Generated the P12 file 'cert/jetty.p12'

>>> ACTION: Converting the P12 file to JKS format:
SOURCE
  cert/jetty.p12
TARGET
  cert/jetty.jks

Importing keystore cert/jetty.p12 to cert/jetty.jks...

SUCCESS: Generated the internal jetty cert file 'cert/jetty.jks'

shell> sudo chown -R dashboard: cert

Or:

shell> mkdir cert
shell> cp /etc/letsencrypt/live/dashdev.continuent.com/fullchain.pem cert/ 
shell> sudo cp /etc/letsencrypt/live/dashdev.continuent.com/privkey.pem cert/
shell> sudo chmod a+r cert/privkey.pem
shell> ./letsencrypt2jetty.pl -f cert/fullchain.pem -k cert/privkey.pem

Keystore file's password: tungsten
Creating `./cert/`
>>> ACTION: Converting the Let's Encrypt source files to P12 format:
SOURCE:
  ./fullchain.pem
  ./privkey.pem
TARGET
  cert/jetty.p12

SUCCESS: Generated the P12 file 'cert/jetty.p12'

>>> ACTION: Converting the P12 file to JKS format:
SOURCE
  cert/jetty.p12
TARGET
  cert/jetty.jks

Importing keystore cert/jetty.p12 to cert/jetty.jks...

SUCCESS: Generated the internal jetty cert file 'cert/jetty.jks'

collect-diag.pl is a script designed to collect all the necessary information, such as logs and files about the running installation.

The resulting file can then be supplied to Continuent Support to assist with issues that you may be having.

When deploying through Kubernetes there are two options available, these are as follows:

...
 api-ssl: true
 api-cert: "<base64-encoded-certificate>"
 ssl: false
...

...
 api-ssl: true
 api-cert: "<base64-encoded-certificate>"
 ssl: true
 cert: "<base64-encoded-certificate>"
 hostname-validation: false
...

These actions can be used on standalone or composite services. They include cctrl operations like heartbeat, recover, switch and failover as well as changing the cluster's policy.

To change the policy for a service, click on the current policy displayed in the service header and then click on the desired new policy in the dropdown. In a composite cluster, you can set a different policy for each service individually.

To execute service operations do the following:

Figure 9.1. Service Policy Dropdown

Figure 9.2. Cluster Lock Dropdown

Figure 9.3. Composite Service Operations

Figure 9.4. Service Operartions

To execute a host operation, please follow the following steps:

Figure 9.5. Primary Host Operations

Figure 9.6. Replica Host Operations

Once an action is clicked on, the operation will lock the service and block it (See Figure 9.7, “Blocked Cluster”) from all Dashboard users for the duration of the operation. By default the Dashboard has a 120-second timeout for operations. Please bear in mind that some actions may take much longer, especially with large clusters. For these use-cases it is recommended to adjust the rest_timeout property in the config.json file.

Figure 9.7. Blocked Cluster

In addition to the control operations listed in the previous sections there are several operations unique to the Dashboard and it's interaction with Tungsten Clusters.

Additionally for dataservices you can use "Lock" & "Unlock" operations from the lock dropdown next to the service actions. (See Figure 9.3, “Composite Service Operations”) This prevents other Dashboard users from executing operations on the service without first unlocking the cluster by hand.

Figure 9.8. Cluster Edit Form

Question: I lost my password and got locked out of the Dashboard

Answer:

To recover access to a single account, and you still have Dashboard access via another user, resolve by recreating the Dashboard account.

If you no longer have access to the Dashboard from any account, you can use the fact that if the users.json file does not exist at startup, the Dashboard will recreate the default user for you as long as the default admin user and password environment variables exist.

Perform the following steps to remove all user accounts and revert to the default admin user:

When the user file does not exist the Dashboard will recreate the default user for you when it restarts, use this to recover access

Problem : You receive a 404 after "test-connection" was successful

This situation happens when the Dashboard cannot resolve the domain names from returned by the API endpoint api/v8/manager/cluster/topology

For example:

{
    "payloadType": "ClusterTopologyPayload",
    "payloadVersion": "1",
    "payload": {
        "topology": "CLUSTERED_PRIMARY",
        "name": "oulu",
        "datasources": [
            {
                "name": "db21",
                /* the value of "host" must be resolvable by Dashboard's dns */
                "host": "db21", 
                "role": "master"
            },
            {
                "name": "db22",
                /* the value of "host" must be resolvable by Dashboard's dns */
                "host": "db22",
                "role": "slave"
            },
            {
                "name": "db23",
                /* the value of "host" must be resolvable by Dashboard's dns */
                "host": "db23",
                "role": "witness"
            }
        ],
        "routers": [...]
    }
}

If the Dashboard cannot obtain an IP address for the node name located above, the load balancing functionality breaks down. To resolve the issue:

Make sure that all of the hosts returned by the API topology endpoint can be resolved before proceeding further. Once all hostnames are resolvable, re-add the cluster to the Dashboard or reload the topology using the reload action or by making a small change to it, such as changing the first host.

Example Scenario

The Dashboard is running but you cannot stop it using docker compose down because you inadvertantly removed the original installation directory.

In this scenario you will need to clean up and re-install.

Is everything removed from a docker?

shell> docker image ls | grep dashboard
shell> docker container ls -a | grep dashboard
shell> docker volume ls | grep dashboard
shell> docker network ls | grep dashboard

Depending upon the outputs from above, you can then user the following command to clean up:

shell> docker [component] rm [component-id]

Check if there are no environment leftovers:

shell> env | grep DASHBOARD

See Chapter 3, Installation for more details on installation.

During the very first installation you will be prompted:

Unique dashboard secret is not defined in .env file. Do you want to generate it? [Y/n]:
DASHBOARD_SECRET=hOYSFI7oOYaRjapPvOXPK7IazH1osZk0
generated and added to .env file.
Please store this secret in a safe place.
You will need it for future upgrades of the Dashboard.

As a message says, store the DASHBOARD_SECRET string somewhere safe.

Data (especially passwords) in persistent volumes in a Docker type of installation are encrypted using this DASHBOARD_SECRET. Losing DASHBOARD_SECRET when doing updates or upgrades will lead to losing your encrypted data.

During updates (from an already configured directory), you won't be prompted for this secret string.

During upgrades (from a new release directory), the install script will prompt you to provide the DASHBOARD_SECRET.

shell> cat .env | grep DASHBOARD_SECRET

shell> echo $DASHBOARD_SECRET

shell> kubectl -n tungsten-dashboard get secret tungsten-dashboard-secret --template={{.data.key}} | base64 --decode

Connect to a running container using a command:

shell> docker exec -it tungsten-dashboard-service-1 /bin/bash

Eventually, you can find the proper dashbaord service name using:

shell> docker container ls

Inside a container you can check persistent data using:

shell> ls -l persistent
shell> cat persistent/config.json
shell> cat persistent/users.json
shell> cat persistent/clusters.json