Monitoring Tungsten Clusters Using Grafana
The below example procedure is designed to help you get Grafana installed and working with the goal of monitoring Tungsten Clusters through the Dashboard.
This section of the documentation is a summary guide for how to install an external software product, Grafana. The usual caveats apply, and as always, your mileage may vary.
For more information about getting started with Grafana, please visit the Grafana website
Example Grafana Installation Procedure
This procedure example uses the YUM-based method. For other ways to install Grafana, please visit the Grafana install page
First, create the YUM repository configuration file for Grafana:
shell> sudo vi /etc/yum.repos.d/grafana.repo
[grafana]
name=grafana
baseurl=https://packages.grafana.com/oss/rpm
repo_gpgcheck=1
enabled=1
gpgcheck=1
gpgkey=https://packages.grafana.com/gpg.key
sslverify=1
sslcacert=/etc/pki/tls/certs/ca-bundle.crt
Install Grafana using yum:
shell> sudo yum install grafana
Example Grafana Configuration Procedure
REQUIRED STEP - Configure Embedding
In order to use the Grafana integration with the Tungsten Dashboard, one line needs to be added to the [security] stanza in the Grafana configuration file
(normally /etc/grafana/grafana.ini). This setting is usually commented out and set to false, so just add a new line under the commented one:
shell> sudo vi /etc/grafana/grafana.ini
...
[security]
;allow_embedding = false
allow_embedding = true
...
OPTIONAL STEP - Configure Anonymous Auth
The embedded Grafana panel will require a login and password. To disable this requirement, and allow the panels to be shown without authentication via limited read-only
access, add two lines under the [auth.anonymous] stanza in the Grafana configuration file (normally /etc/grafana/grafana.ini). These settings are
usually commented out, so just add the new lines under the commented ones:
shell> sudo vi /etc/grafana/grafana.ini
...
[auth.anonymous]
;enabled = false
enabled = true
;org_name = Main Org.
org_name = {Your Exact Organization Name In Grafana}
...
OPTIONAL STEP - Configure HTTPS
The embedded Grafana panel will be called on whatever transport is used to load the main Tungsten Dashboard page, i.e. http or https. To configure Grafana to use https, add
four lines to the [server] stanza in the Grafana configuration file (normally /etc/grafana/grafana.ini). These settings are usually commented out,
so just add the new lines under the commented ones:
shell> sudo vi /etc/grafana/grafana.ini
...
[server]
;protocol = http
protocol = https
;domain = localhost
domain = dashboard.yourdomain.com
;cert_file =
cert_file = /etc/letsencrypt/archive/dashboard.yourdomain.com/fullchain1.pem
;cert_key =
cert_key = /etc/letsencrypt/archive/dashboard.yourdomain.com/privkey1.pem
...
It is critical that the domain = value be the same FQDN as the one the Dashboard web service answers to, and that the same keys are in use.
Our example shows Let's Encrypt certificates that are shared with the Dashboard web server instance. For this to work, the permissions of the key files must allow for Grafana to access them. Below is an example of how you could allow access to the needed certificate files:
shell> sudo chgrp grafana /etc/letsencrypt/archive/dashboard.yourdomain.com/privkey1.pem
shell> sudo chmod g+r /etc/letsencrypt/archive/dashboard.yourdomain.com/privkey1.pem
Please remember to restart Grafana when the certificates expire and get renewed!
Below is a sample Let's Encrypt command to get a cert for dashboard.yourdomain.com, assuming that there is real DNS for that domain, that it resolves for the world, and that web server is reachable by the world:
shell> certbot certonly \
--webroot \
--renew-by-default \
--agree-tos \
-v \
--debug \
--email you@yourdomain.com \
-w /volumes/data/tungsten/html \
-d dashboard.yourdomain.com \
--dry-run
Please remember to remove the --dry-run argument at the end and re-run to get the real certs!
All examples provided for convenience. As always, YMMV, and supporting Grafana and/or certificates is outside Continuent's scope.
Example Grafana Boot Configuration Procedure
The YUM-based install automatically creates the grafana user, along with the systemd and init.d boot scripts. This means you do not
have to create the boot scripts by hand!
init.d-based procedure
Enable the
grafana-serverservice to start at boot time viachkconfig, and then start it usingservice:shell> sudo chkconfig --add grafana-servershell> sudo chkconfig --list | grep grafana-servershell> sudo service grafana-server startshell> sudo service grafana-server statussystemd-based procedure
Use
systemctlto reload the boot config, enable thegrafana-serverservice to start at boot time, and then start Grafana:shell> sudo systemctl daemon-reloadshell> sudo systemctl enable grafana-servershell> sudo systemctl start grafana-server
Example Grafana Test Procedure
Once the Grafana server has been started, you may test that it is running via browser URL http://{yourServer}:3000
Login as user admin with a password of admin, and please change the admin password when prompted to do so.
Grafana may now be added to the Dashboard via the config.json file in the Dashboard WEBROOT directory. Add the configuration option "enableGrafana":1 and
refresh the Dashboard page in the browser to see the additional button in the top navigation bar.
For more information about next steps with Grafana, please visit the Grafana website
Example Grafana Setup and Usage
Once logged into the Grafana server as admin, you may configure a data source and import the dashboards.
- Create a data source using Prometheus
- Click the Configuration cog on the left nav bar
- Click "Add data source"
- Choose prometheus, then add 'http://localhost:9090' to the HTTP URL field
- Click Save & Test at the bottom. This should create a new data source named 'Prometheus' for use a few steps below.
- Optional Step: Import the Included Prometheus and Grafana Dashboards
- If you want to use the built-in metrics for Prometheus and/or Grafana, import the included dashboards as desired.
- Click on the Dashboards Tab in the center window to the right of the Settings Tab, then click on the blue Import button for each of Prometheus Stats, Prometheus 2.0 Stats and Grafana metrics.
- Import the Continuent Tungsten Dashboard
- Hover over the Dashboards icon in the left nav bar, then select Manage from the sub-menu. Click the Import link to the right of the green New Dashboard button.
- In the Grafana.com Dashboard field, enter
12760for the Continuent Tungsten dashboard, then click Load. - Select the Prometheus data source, then click the green Import button.
- If you have Prometheus setup correctly and running, you should see results instantly.
- Save this Dashboard by clicking the 3.5 inch floppy icon in the upper-right corner, then click the green Save button.
- Click the star in the upper-right corner to make this dashboard a favorite. This makes finding the dashboard MUCH easier.
- Import the Node Exporter Full Dashboard
- Hover over the Dashboards icon in the left nav bar, then select Manage from the sub-menu. Click the Import link to the right of the green New Dashboard button.
- In the Grafana.com Dashboard field, enter
1860for the Node Exporter Full dashboard, then click Load. - Select the Prometheus data source, then click the green Import button.
- If you have Prometheus setup correctly and running, you should see results instantly.
- Save this Dashboard by clicking the 3.5 inch floppy icon in the upper-right corner, then click the green Save button.
- Click the star in the upper-right corner to make this dashboard a favorite. This makes finding the dashboard MUCH easier.
- Import the Percona MySQL Dashboard
- Hover over the Dashboards icon in the left nav bar, then select Manage from the sub-menu. Click the Import link to the right of the green New Dashboard button.
- In the Grafana.com Dashboard field, enter
7362for the Percona MySQL dashboard, then click Load. - Select the Prometheus data source, then click the green Import button.
- If you have Prometheus setup correctly and running, you should see results instantly.
- Save this Dashboard by clicking the 3.5 inch floppy icon in the upper-right corner, then click the green Save button.
- Click the star in the upper-right corner to make this dashboard a favorite. This makes finding the dashboard MUCH easier.
For more information about next steps with Grafana, please visit the Grafana website