Chapter 6. Operations Guide

Table of Contents

6.1. The Home Directory
6.2. Establishing the Shell Environment
6.3. Checking Dataservice Status
6.3.1. Latency or Relative Latency Display
6.3.2. Getting Detailed Information
6.3.3. Understanding Datasource Roles
6.3.4. Understanding Datasource States
6.3.5. Understanding Replicator Roles
6.3.6. Changing Datasource States
6.3.7. Datasource Statuses
6.3.8. Datasource States and Policy Mode Interactions
6.4. Policy Modes
6.4.1. Setting Policy Modes
6.5. Switching Primary Hosts
6.5.1. Automatic Primary Failover
6.5.2. Manual Primary Switch
6.6. Datasource Recovery Steps
6.6.1. Recover a failed Replica
6.6.2. Recover a failed Primary
6.7. Composite Cluster Switching, Failover and Recovery
6.7.1. Composite Cluster Site Switch
6.7.2. Composite Cluster Site Failover (Forced Switch)
6.7.3. Composite Cluster Site Recovery
6.7.4. Composite Cluster Relay Recovery
6.8. Composite Active/Active Recovery
6.9. Managing Transaction Failures
6.9.1. Identifying a Transaction Mismatch
6.9.2. Skipping Transactions
6.10. Creating a Backup
6.10.1. Using a Different Backup Tool
6.10.2. Automating Backups
6.10.3. Using a Different Directory Location
6.10.4. Creating an External Backup
6.11. Restoring a Backup
6.11.1. Restoring a Specific Backup
6.11.2. Restoring an External Backup
6.11.3. Restoring from Another Replica
6.11.4. Manually Recovering from Another Replica
6.11.5. Reprovision a MySQL Replica using rsync
6.11.6. Rebuilding a Lost Datasource
6.11.7. Resetting an Entire Dataservice from Filesystem Snapshots
6.12. Migrating and Seeding Data
6.12.1. Migrating from MySQL Native Replication 'In-Place'
6.12.2. Migrating from MySQL Native Replication Using a New Service
6.13. Resetting a Tungsten Cluster Dataservice
6.13.1. Reset a Single Site in a Multi-Site/Active-Active Topology
6.13.2. Reset All Sites in a Multi-Site/Active-Active topology
6.14. Replicator Fencing
6.14.1. Fencing a Replica Node Due to a Replication Fault
6.14.2. Fencing Primary Replicators
6.15. Performing Database or OS Maintenance
6.15.1. Performing Maintenance on a Single Replica
6.15.2. Performing Maintenance on a Primary
6.15.3. Performing Maintenance on an Entire Dataservice
6.15.4. Making Schema Changes
6.15.5. Upgrading or Updating your JVM
6.16. Performing Tungsten Cluster Maintenance
6.17. Monitoring Tungsten Cluster
6.17.1. Managing Log Files with logrotate
6.17.2. Monitoring Status Using cacti
6.17.3. Monitoring Status Using nagios
6.18. Rebuilding THL on the Primary

Tungsten Cluster™ has a wide range of tools and functionality available for checking and managing the status of a dataservice. The majority of the management and information structure is based around a small number of command-line utilities that provide a complete range of tools and information, either through a direct command-line, or secondary shell like interface.

When installing the dataservice using tpm , if requested, the login script for the staging user (for example .bashrc ) will have been updated to execute a script within the installation directory called . This configures the location of the installation, configuration, and adds the script and binary directories to the PATH so that the commands can be executed without having to use the full path to the tools.

If the script was not added to the login script automatically, or needs to be added to the current session, the script is located within the share directory of the installation directory. For example, /opt/continuent/share/ To load into the current session use source. See Section 6.2, “Establishing the Shell Environment” for more information.

shell> source /opt/continuent/share/

The main tool for controlling dataservices is cctrl. This provides a shell like interface for querying and managing the dataservice and includes shell-like features such as command history and editing. Commands can be executed using cctrl either interactively:

shell> cctrl
connect to 'alpha@host1'
alpha: session established
[LOGICAL:EXPERT] /alpha > ls

Or by supplying a command and piping that as input to the cctrl shell:

shell> echo 'ls' | cctrl