Tungsten Clustering

Installing via INI File with CA-Signed Certificates

This procedure will take a signed certificate from a known Certificate Authority and use it as the basis for all SSL operations within the cluster, not including Connector client-server SSL, which is configured separately. Please visit "Configuring Connector SSL" for more information about configuring Connector SSL.
The below example procedure assumes that you have an existing, installed and running cluster with security enabled by setting
disable-security-controls=false
Assume a 3-node cluster called alpha with member hosts db1, db2 and db3.
Warning
In all examples below, because you are updating an existing secure installation, the password tungsten is required, do not change it.

Select one node to create the proper set of certs, i.e. db1:

shell> su - tungsten
shell> mkdir /etc/tungsten/secure
shell> mkdir ~/certs
shell> cd ~/certs

Copy the available files (CA cert, Intermediate cert (if needed), signed cert and signing key) into ~/certs/, i.e.:
```
ca.crt.pem
int.crt.pem
signed.crt.pem
signing.key.pem
```
Create a pkcs12 (.p12) version of the signed certificate:
```
shell> openssl pkcs12 -export -in ~/certs/signed.crt.pem -inkey ~/certs/signing.key.pem \
-out ~/certs/tungsten_sec.crt.p12 -name mysql
Enter Export Password: tungsten
Verifying - Enter Export Password: tungsten
```
Important
When using OpenSSL 3.0 with Java 1.8, you MUST add the -legacy option to the openssl command.
Important
If you choose a different alias name rather than mysql shown in the example above, then you must also specify the tpm option java-mysql-alias in your /etc/tungsten/tungsten.ini

Create a pkcs12-based keystore (.jks) version of the signed certificate:

shell> keytool -importkeystore -deststorepass tungsten -destkeystore /etc/tungsten/secure/tungsten_keystore.jks \
-srckeystore ~/certs/tungsten_sec.crt.p12 -srcstoretype pkcs12 -deststoretype pkcs12
Importing keystore /home/tungsten/certs/tungsten_sec.crt.p12 to /etc/tungsten/secure/tungsten_keystore.jks...
Enter source keystore password:  tungsten
Entry for alias replserver successfully imported.
Import command completed:  1 entries successfully imported, 0 entries failed or cancelled

Import the Certificate Authority's certificate into the keystore:

shell> keytool -import -alias mysql -file ~/certs/ca.crt.pem -keypass tungsten \
-keystore /etc/tungsten/secure/tungsten_keystore.jks -storepass tungsten
...
Trust this certificate? [no]:  yes
Certificate was added to keystore

Import the Certificate Authority's intermediate certificate (if supplied) into the keystore:

shell> keytool -import -alias mysql -file ~/certs/int.crt.pem -keypass tungsten \
-keystore /etc/tungsten/secure/tungsten_keystore.jks -storepass tungsten
Certificate was added to keystore

Export the cert from the keystore into file client.cer for use in the next step to create the truststore:

shell> keytool -export -alias mysql -file ~/certs/client.cer \
-keystore /etc/tungsten/secure/tungsten_keystore.jks
Enter keystore password:  tungsten
Certificate stored in file </home/tungsten/certs/client.cer>

Create the truststore:

shell> keytool -import -trustcacerts -alias mysql -file ~/certs/client.cer \
-keystore /etc/tungsten/secure/tungsten_truststore.ts -storepass tungsten -noprompt
Certificate was added to keystore

Create the rmi_jmx password store entry:

shell> tpasswd -c tungsten tungsten -t rmi_jmx -p /etc/tungsten/secure/passwords.store -e \
-ts /etc/tungsten/secure/tungsten_truststore.ts -tsp tungsten
Using parameters: 
-----------------
security.properties      = /opt/continuent/tungsten/cluster-home/../cluster-home/conf/security.properties
password.file.location     = /etc/tungsten/secure/passwords.store
encrypted.password     = true
truststore.location      = /etc/tungsten/secure/tungsten_truststore.ts
truststore.password      = *********
-----------------
Creating non existing file: /etc/tungsten/secure/passwords.store
User created successfuly: tungsten

Create the tls password store entry:

shell> tpasswd -c tungsten tungsten -t unknown -p /etc/tungsten/secure/passwords.store -e \
-ts /etc/tungsten/secure/tungsten_truststore.ts -tsp tungsten
Using parameters: 
-----------------
security.properties      = /opt/continuent/tungsten/cluster-home/../cluster-home/conf/security.properties
password.file.location     = /etc/tungsten/secure/passwords.store
encrypted.password     = true
truststore.location      = /etc/tungsten/secure/tungsten_truststore.ts
truststore.password      = ********
-----------------
User created successfuly: tungsten

List and verify the user for each security service password store entry, rmi_jmx and tls (which has a display tag of unknown):

shell> tpasswd -l -p /etc/tungsten/secure/passwords.store -ts /etc/tungsten/secure/tungsten_truststore.ts
Using parameters: 
-----------------
security.properties      = /opt/continuent/tungsten/cluster-home/../cluster-home/conf/security.properties
password.file.location     = ./passwords.store
encrypted.password     = true
truststore.location      = ./tungsten_truststore.ts
truststore.password      = ********
-----------------
Listing users by application type:

[unknown]
-----------
tungsten

[rmi_jmx]
-----------
tungsten

On host db1, transfer the generated certificates to the same path on all remaining hosts:

shell> for host in `seq 2 3`; do rsync -av /etc/tungsten/secure/ db${host}:/etc/tungsten/secure/; done

Edit the /etc/tungsten/tungsten.ini configuration file on all nodes and add:
Example tungsten.ini
```
[defaults]
...
disable-security-controls=false
java-keystore-path=/etc/tungsten/secure/tungsten_keystore.jks
java-keystore-password=tungsten
java-truststore-path=/etc/tungsten/secure/tungsten_truststore.ts
java-truststore-password=tungsten
rmi-ssl=true
rmi-authentication=true
rmi-user=tungsten
java-passwordstore-path=/etc/tungsten/secure/passwords.store
```
Show argument definitions
disable-security-controls=falseDisables all forms of security, including SSL, TLS and authentication.
java-keystore-path=/etc/tungsten/secure/tungsten_keystore.jksLocal path to the Java Keystore file. Specific for intra cluster communication. NOTE: When java-keystore-path is passed to tpm, the keystore must contain both tls and mysql certs when appropriate. tpm will NOT add mysql cert nor generate tls cert when this flag is found, so both certs must be manually imported already. Additionally,java-mysql-alias must be specificed when using custom keystores.
java-keystore-password=tungstenSet the password for unlocking the tungsten_keystore.jks file in the security directory. Specific for intra cluster communication.
java-truststore-path=/etc/tungsten/secure/tungsten_truststore.tsLocal path to the Java Truststore file.
java-truststore-password=tungstenThe password for unlocking the tungsten_truststore.jks file in the security directory.
rmi-ssl=trueEnable SSL encryption of RMI communication on this host.
rmi-authentication=trueEnable RMI authentication for the services running on this host.
rmi-user=tungstenThe username for RMI authentication.
java-passwordstore-path=/etc/tungsten/secure/passwords.storeLocal path to the Java Password Store file
Important
If you chose a different alias name rather than mysql shown in the examples, then you must also specify the java-mysql-alias
Important
When java-keystore-path is passed to tpm, the keystore must contain both tls and mysql certs when appropriate. tpm will NOT add mysql cert nor generate tls cert when this flag is found, so both certs must be manually imported already.
On one node only, enable MAINTENANCE mode:
```
cctrl> set policy maintenance
```
On ALL nodes, stop the tungsten software, execute the update, then restart:
Warning
This procedure requires the complete restart of all layers and will cause a brief outage.
```
shell> tpm query staging
shell> cd {staging_dir}
shell> stopall
shell> tools/tpm update --replace-release
shell> startall
```

On one node only, enable AUTOMATIC mode and check cluster status:

shell> cctrl

cctrl> set policy automatic
cctrl> ls
COORDINATOR[db1:AUTOMATIC:ONLINE]

ROUTERS:
+---------------------------------------------------------------------------------+
|connector@db1[9871](ONLINE, created=0, active=0)                                 |
|connector@db2[27930](ONLINE, created=0, active=0)                                |
|connector@db3[23727](ONLINE, created=0, active=0)                                |
+---------------------------------------------------------------------------------+

DATASOURCES:
+---------------------------------------------------------------------------------+
|db1(master:ONLINE, progress=1, THL latency=0.656)                                |
|STATUS [OK] [2019/06/06 12:48:11 PM UTC]                                         |
+---------------------------------------------------------------------------------+
|  MANAGER(state=ONLINE)                                                          |
|  REPLICATOR(role=master, state=ONLINE)                                          |
|  DATASERVER(state=ONLINE)                                                       |
|  CONNECTIONS(created=0, active=0)                                               |
+---------------------------------------------------------------------------------+
+---------------------------------------------------------------------------------+
|db2(slave:ONLINE, progress=1, latency=9.858)                                     |
|STATUS [OK] [2019/06/06 12:48:11 PM UTC]                                         |
+---------------------------------------------------------------------------------+
|  MANAGER(state=ONLINE)                                                          |
|  REPLICATOR(role=slave, master=db1, state=ONLINE)                               |
|  DATASERVER(state=ONLINE)                                                       |
|  CONNECTIONS(created=0, active=0)                                               |
+---------------------------------------------------------------------------------+
+---------------------------------------------------------------------------------+
|db3(slave:ONLINE, progress=1, latency=19.235)                                    |
|STATUS [OK] [2019/06/06 12:48:10 PM UTC]                                         |
+---------------------------------------------------------------------------------+
|  MANAGER(state=ONLINE)                                                          |
|  REPLICATOR(role=slave, master=db1, state=ONLINE)                               |
|  DATASERVER(state=ONLINE)                                                       |
|  CONNECTIONS(created=0, active=0)                                               |
+---------------------------------------------------------------------------------+