Continuent Documentation

Manual Home
Tungsten Clustering (for MySQL) 7.1 Manual
Breadcrumbs
Chapters
Preface
1. Introduction
2. Deployment
3. Deployment: MySQL Topologies
4. Deployment: Advanced
5. Deployment: Security
6. Operations Guide
7. Tungsten Connector
8. Tungsten Manager
9. Command-line Tools
10. The tpm Deployment Command
11. Tungsten REST API (APIv2)
12. Replication Filters
13. Performance, Tuning and Testing
A. Release Notes
B. Prerequisites
C. Troubleshooting
D. Files, Directories, and Environment
E. Terminology Reference
F. Internals
G. Frequently Asked Questions (FAQ)
H. Ecosystem Support
I. Configuration Property Reference
Tungsten Clustering (for MySQL) 7.1 Manual
    Next

Tungsten Clustering (for MySQL) 7.1 Manual

Continuent Ltd

Copyright © 2023 Continuent Ltd

Abstract

This manual documents Tungsten Clustering (for MySQL), a high performance, High Availability and Disaster Recovery for MySQL clustering.

This manual includes information for 7.1, up to and including 7.1.4.

Build date: 2024-10-10 (1408c26a)

Up to date builds of this document: Tungsten Clustering (for MySQL) 7.1 Manual (Online), Tungsten Clustering (for MySQL) 7.1 Manual (PDF)

Table of Contents

Preface
1. Legal Notice
2. Conventions
3. Quickstart Guide
1. Introduction
1.1. Tungsten Replicator
1.1.1. Transaction History Log (THL)
1.2. Tungsten Manager
1.3. Tungsten Connector
2. Deployment
2.1. Host Types
2.1.1. Manager Hosts
2.1.2. Connector (Router) Hosts
2.1.3. Replicator Hosts
2.1.4. Active Witness Hosts
2.2. Requirements
2.2.1. Operating Systems Support
2.2.2. Database Support
2.2.3. RAM Requirements
2.2.4. Disk Requirements
2.2.5. Java Requirements
2.2.6. Cloud Deployment Requirements
2.2.7. Docker Support Policy
2.3. Deployment Sources
2.3.1. Using the TAR/GZipped files
2.3.2. Using the RPM package files
2.4. Common tpm Options During Deployment
2.5. Best Practices
2.5.1. Best Practices: Deployment
2.5.2. Best Practices: Upgrade
2.5.3. Best Practices: Operations
2.5.4. Best Practices: Maintenance
3. Deployment: MySQL Topologies
3.1. Deploying Standalone HA Clusters
3.1.1. Prepare: Standalone HA Cluster
3.1.2. Install: Standalone HA Cluster
3.1.3. Best Practices: Standalone HA Cluster
3.2. Deploying Composite Active/Passive Clustering
3.2.1. Prepare: Composite Active/Passive Cluster
3.2.2. Install: Composite Active/Passive Cluster
3.2.3. Best Practices: Composite Active/Passive Cluster
3.2.4. Adding a remote Composite Cluster
3.3. Deploying Multi-Site/Active-Active Clustering
3.3.1. Prepare: Multi-Site/Active-Active Clusters
3.3.2. Install: Multi-Site/Active-Active Clusters
3.3.3. Best Practices: Multi-Site/Active-Active Clusters
3.3.4. Configuring Startup on Boot
3.3.5. Resetting a single dataservice
3.3.6. Resetting all dataservices
3.3.7. Provisioning during live operations
3.3.8. Adding a new Cluster/Dataservice
3.3.9. Enabling SSL for Replicators Only
3.3.10. Dataserver maintenance
3.4. Deploying Composite Active/Active Clusters
3.4.1. Prepare: Composite Active/Active Clusters
3.4.2. Install: Composite Active/Active Clusters
3.4.3. Best Practices: Composite Active/Active Clusters
3.4.4. Configuring Startup on Boot
3.4.5. Resetting a single dataservice
3.4.6. Resetting all dataservices
3.4.7. Dataserver maintenance
3.4.8. Adding a Cluster to a Composite Active/Active Topology
3.5. Deploying Composite Dynamic Active/Active
3.5.1. Enabling Composite Dynamic Active/Active
3.6. Deploying Tungsten Connector Only
3.7. Deploying Additional Datasources, Managers, or Connectors
3.7.1. Adding Datasources to an Existing Deployment
3.7.2. Adding Active Witnesses to an Existing Deployment
3.7.3. Replacing an Active Witness as a Full Cluster Node
3.7.4. Replacing a Full Cluster Node as an Active Witness
3.7.5. Adding Connectors to an Existing Deployment
3.7.6. Converting from a single cluster to a composite cluster
3.8. Replicating Data Into an Existing Dataservice
3.9. Replicating Data Out of a Cluster
3.9.1. Prepare: Replicating Data Out of a Cluster
3.9.2. Deploy: Replicating Data Out of a Cluster
3.10. Replicating from a Cluster to a Datawarehouse
3.10.1. Replicating from a Cluster to a Datawarehouse - Prerequisites
3.10.2. Replicating from a Cluster to a Datawarehouse - Configuring the Cluster Nodes
3.10.3. Replicating from a Cluster to a Datawarehouse - Configuring the Cluster-Extractor
3.11. Migrating and Seeding Data
3.11.1. Migrating from MySQL Native Replication 'In-Place'
3.11.2. Migrating from MySQL Native Replication Using a New Service
3.11.3. Seeding Data through MySQL
4. Deployment: Advanced
4.1. Deploying Parallel Replication
4.1.1. Application Prerequisites for Parallel Replication
4.1.2. Enabling Parallel Apply During Install
4.1.3. Channels
4.1.4. Parallel Replication and Offline Operation
4.1.5. Adjusting Parallel Replication After Installation
4.1.6. Monitoring Parallel Replication
4.1.7. Controlling Assignment of Shards to Channels
4.1.8. Disk vs. Memory Parallel Queues
4.2. Distributed Datasource Groups
4.2.1. Introduction to DDG
4.2.2. How DDG Works
4.2.3. Configuring DDG
4.3. Starting and Stopping Tungsten Cluster
4.3.1. Restarting the Replicator Service
4.3.2. Restarting the Connector Service
4.3.3. Restarting the Manager Service
4.3.4. Restarting the Multi-Site/Active-Active Replicator Service
4.4. Configuring Startup on Boot
4.4.1. Configuring Multi-Site/Active-Active Replicator Startup on Boot
4.5. Upgrading Tungsten Cluster
4.5.1. Upgrading using the Staging Method (with ssh Access)
4.5.2. Upgrading when using INI-based configuration, or without ssh Access
4.5.3. Upgrade/Convert: From Multi-Site/Active-Active (MSAA) to Composite Active/Passive (CAP)
4.5.4. Upgrade/Convert: From Multi-Site/Active-Active (MSAA) to Composite Active/Active (CAA)
4.5.5. Installing an Upgraded JAR Patch
4.5.6. Installing Patches
4.5.7. Upgrading to v7.0.0+
4.6. Removing Datasources, Managers or Connectors
4.6.1. Removing a Datasource from an Existing Deployment
4.6.2. Removing a Composite Datasource/Cluster from an Existing Deployment
4.6.3. Removing a Connector from an Existing Deployment
5. Deployment: Security
5.1. Enabling Security
5.1.1. Enabling Security using the Staging Method
5.1.2. Enabling Security using the INI Method
5.2. Disabling Security
5.3. Creating Suitable Certificates
5.3.1. Creating Tungsten Internal Certificates Using tpm cert
5.3.2. Creating Tungsten Internal Certificates Manually
5.4. Installing from a Staging Host with Custom Certificates
5.4.1. Installing from a Staging Host with Manually-Generated Certificates
5.4.2. Installing from a Staging Host with Certificates Generated by tpm cert
5.5. Installing via INI File with Custom Certificates
5.5.1. Installing via INI File with Manually-Generated Certificates
5.5.2. Installing via INI File with Certificates Generated by tpm cert
5.6. Installing via INI File with CA-Signed Certificates
5.7. Replacing the JGroups Certificate from a Staging Directory
5.8. Replacing the TLS Certificate from a Staging Directory
5.9. Removing JGroups Encryption from a Staging Directory
5.10. Removing JGroups Encryption via INI File
5.11. Removing TLS Encryption from a Staging Directory
5.12. Removing TLS Encryption via INI File
5.13. Enabling Tungsten<>Database Security
5.13.1. Enabling Database SSL
5.13.2. Configure Tungsten<>Database Secure Communication
5.13.3. Configuring Connector SSL
6. Operations Guide
6.1. The Home Directory
6.2. Establishing the Shell Environment
6.3. Checking Dataservice Status
6.3.1. Latency or Relative Latency Display
6.3.2. Getting Detailed Information
6.3.3. Understanding Datasource Roles
6.3.4. Understanding Datasource States
6.3.5. Understanding Replicator Roles
6.3.6. Changing Datasource States
6.3.7. Datasource Statuses
6.3.8. Datasource States and Policy Mode Interactions
6.4. Policy Modes
6.4.1. Setting Policy Modes
6.5. Switching Primary Hosts
6.5.1. Automatic Primary Failover
6.5.2. Manual Primary Switch
6.6. Datasource Recovery Steps
6.6.1. Recover a failed Replica
6.6.2. Recover a failed Primary
6.7. Composite Cluster Switching, Failover and Recovery
6.7.1. Composite Cluster Site Switch
6.7.2. Composite Cluster Site Failover (Forced Switch)
6.7.3. Composite Cluster Site Recovery
6.7.4. Composite Cluster Relay Recovery
6.8. Composite Active/Active Recovery
6.9. Managing Transaction Failures
6.9.1. Identifying a Transaction Mismatch
6.9.2. Skipping Transactions
6.10. Creating a Backup
6.10.1. Using a Different Backup Tool
6.10.2. Automating Backups
6.10.3. Using a Different Directory Location
6.10.4. Creating an External Backup
6.11. Restoring a Backup
6.11.1. Restoring a Specific Backup
6.11.2. Restoring an External Backup
6.11.3. Restoring from Another Replica
6.11.4. Manually Recovering from Another Replica
6.11.5. Reprovision a MySQL Replica using rsync
6.11.6. Rebuilding a Lost Datasource
6.11.7. Resetting an Entire Dataservice from Filesystem Snapshots
6.12. Migrating and Seeding Data
6.12.1. Migrating from MySQL Native Replication 'In-Place'
6.12.2. Migrating from MySQL Native Replication Using a New Service
6.13. Resetting a Tungsten Cluster Dataservice
6.13.1. Reset a Single Site in a Multi-Site/Active-Active Topology
6.13.2. Reset All Sites in a Multi-Site/Active-Active topology
6.14. Replicator Fencing
6.14.1. Fencing a Replica Node Due to a Replication Fault
6.14.2. Fencing Primary Replicators
6.15. Performing Database or OS Maintenance
6.15.1. Performing Maintenance on a Single Replica
6.15.2. Performing Maintenance on a Primary
6.15.3. Performing Maintenance on an Entire Dataservice
6.15.4. Making Schema Changes
6.15.5. Upgrading or Updating your JVM
6.15.6. Upgrading between MySQL 5.x and MySQL 8.x
6.16. Performing Tungsten Cluster Maintenance
6.17. Monitoring Tungsten Cluster
6.17.1. Managing Log Files with logrotate
6.17.2. Monitoring Status Using cacti
6.17.3. Monitoring Status Using nagios
6.17.4. Monitoring Status Using Prometheus Exporters
6.18. Rebuilding THL on the Primary
6.19. THL Encryption and Compression
6.19.1. In-Flight Compression
6.19.2. Encryption and Compression On-Disk
7. Tungsten Connector
7.1. Connector/Application Basics
7.1.1. Connector Control Flow
7.2. Basic Connector Configuration
7.3. Clients and Deployment
7.3.1. Connection Pools
7.4. Routing Methods
7.4.1. Connector Routing Methods
7.4.2. Primary/Replica Selection
7.4.3. Connector Load Balancers
7.4.4. Specifying Required Latency
7.4.5. Setting Read Affinity and Direct Reads
7.4.6. Setting Read/Write Affinity in Composite Active/Active Environments
7.4.7. Setting Negative Affinity
7.4.8. Routing using embedded syntax in connect string
7.4.9. Connector Datasource Selection in Composite Clusters
7.4.10. Smartscale Routing
7.4.11. Direct Routing
7.4.12. SQL Routing
7.4.13. Host-based Routing
7.4.14. Port-based Routing
7.4.15. Read-only Routing
7.5. Using Bridge Mode
7.5.1. Enabling Bridge Mode
7.6. User Authentication
7.6.1. user.map File Format
7.6.2. Dual-Paswords and MySQL 8
7.6.3. user.map Direct Routing
7.6.4. user.map Host Options
7.6.5. user.map Updates
7.6.6. Generating user.map Entries from a Script
7.6.7. Encrypting user.map Data
7.6.8. Synchronizing user.map Data
7.6.9. user.map Limitations
7.6.10. Host-based Authentication
7.7. Testing Connectivity Via the Connector
7.7.1. Testing Connectivity in Bridge Mode
7.7.2. Testing Connectivity in Proxy Mode with No R/W Splitting Enabled
7.7.3. Testing Connectivity in Proxy Mode with R/W Splitting Enabled (SmartScale or @direct)
7.8. Connector Operational States
7.8.1. Connections During Automatic Failure/Failover
7.8.2. Connections During Manual Switch
7.8.3. Connections During Connection Failures
7.8.4. Other Errors
7.8.5. Connector Keepalive
7.8.6. Connector Change User as Ping
7.9. Connector/Manager Interface
7.10. Connector Command-Line Interface
7.11. Connector Inline Command Interface
7.11.1. Connector tungsten cluster status Command
7.11.2. Connector tungsten connection count Command
7.11.3. Connector tungsten connection status Command
7.11.4. Connector tungsten flush privileges Command
7.11.5. Connector tungsten gc Command
7.11.6. Connector tungsten help Command
7.11.7. Connector tungsten mem info Command
7.11.8. Connector tungsten show [full] processlist Command
7.11.9. Connector show slave status Command
7.11.10. Connector tungsten show variables Command
7.12. Advanced Configuration
7.12.1. Working with Proxy Protocol v1
7.12.2. Using Multiple Dataservices
7.12.3. Advanced Listeners
7.12.4. Connector Automatic Reconnect
7.12.5. Using the Connector with HA Proxy
7.12.6. Using Fall-Back Bridge Mode
7.12.7. Using the Max Connections Feature
7.12.8. Adjusting the Client Disconnect Delay During Manual Switch
7.12.9. Adjusting the Bridge Mode Forced Client Disconnect Timeout
7.12.10. Adjusting the Connnector Response to Resource Losses
7.12.11. Connector Logging Configuration
7.12.12. Connector Audit Logging
7.12.13. Connector SSL Advertisement Configuration
7.12.14. Connector IP Address Configuration
7.12.15. Deploying a Connector through Docker
7.13. Connector General Limitations
8. Tungsten Manager
8.1. Tungsten Manager Introduction
8.1.1. How Does Failover Work?
8.1.2. Best Practices for Proper Cluster Failovers
8.1.3. Command-Line Monitoring Tools
8.2. Tungsten Manager Failover Behavior
8.2.1. Failover Replication State Scenarios
8.2.2. Recovery Behavior After Failover
8.2.3. Failover Response when MySQL Server Fails
8.2.4. Failover Response when Replica Applier is Latent
8.3. Tungsten Manager Failover Tuning
8.4. Tungsten Manager Failover Internals
8.4.1. Manual Switch Versus Automatic Failover
8.4.2. Switch and Failover Steps for Local Clusters
8.4.3. Switch and Failover Steps for Composite Services
8.5. Tungsten Manager Fault Detection, Fencing and Recovery
8.5.1. Tungsten Manager Definitions
8.5.2. Cluster Monitoring and Notification Events
8.5.3. Rule Organization - Detection, Investigation, Fencing, Recovery
8.6. Cluster State Savepoints
8.7. Adjusting JVM Settings for the Manager
9. Command-line Tools
9.1. The cctrl Command
9.1.1. cctrl Command-line Options
9.1.2. cctrl Modes
9.1.3. cctrl Commands
9.2. The check_tungsten_latency Command
9.3. The check_tungsten_online Command
9.4. The check_tungsten_policy Command
9.5. The check_tungsten_progress Command
9.6. The check_tungsten_services Command
9.7. The clean_release_directory Command
9.8. The cluster_backup Command
9.9. The connector Command
9.10. The ddlscan Command
9.10.1. Optional Arguments
9.10.2. Supported Templates and Usage
9.11. The deployall Command
9.12. The dsctl Command
9.12.1. dsctl get Command
9.12.2. dsctl set Command
9.12.3. dsctl reset Command
9.12.4. dsctl help Command
9.13. env.sh Script
9.14. The load-reduce-check Tool
9.14.1. Generating Staging DDL
9.14.2. Generating Live DDL
9.14.3. Materializing a View
9.14.4. Generating Sqoop Load Commands
9.14.5. Generating Metadata
9.14.6. Compare Loaded Data
9.15. The manager Command
9.16. The materialize Command
9.17. The multi_trepctl Command
9.17.1. multi_trepctl Options
9.17.2. multi_trepctl Commands
9.18. The query Command
9.19. The replicator Command
9.20. The startall Command
9.21. The stopall Command
9.22. The tapi Command
9.23. The thl Command
9.23.1. thl Position Commands
9.23.2. thl dsctl Command
9.23.3. thl list Command
9.23.4. thl tail Command
9.23.5. thl index Command
9.23.6. thl purge Command
9.23.7. thl info Command
9.23.8. thl help Command
9.24. The trepctl Command
9.24.1. trepctl Options
9.24.2. trepctl Global Commands
9.24.3. trepctl Service Commands
9.25. The tmonitor Command
9.26. The tpasswd Command
9.27. The tprovision Script
9.28. The tungsten_find_orphaned Command
9.28. Examples:
9.29. The tungsten_find_position Command
9.30. The tungsten_find_seqno Command
9.31. The tungsten_get_mysql_datadir Script
9.32. The tungsten_get_status Script
9.33. The tungsten_get_ports Script
9.34. The tungsten_health_check Script
9.35. The tungsten_merge_logs Script
9.36. The tungsten_monitor Script
9.37. The tungsten_mysql_ssl_setup Script
9.38. The tungsten_newrelic_event Command
9.39. The tungsten_nagios_backups Command
9.40. The tungsten_nagios_online Command
9.41. The tungsten_post_process Command
9.42. The tungsten_prep_upgrade Script
9.43. The tungsten_provision_thl Command
9.43.1. Provisioning from RDS
9.43.2. tungsten_provision_thl Reference
9.44. The tungsten_purge_thl Command
9.45. The tungsten_reset_manager Command
9.46. The tungsten_send_diag Script
9.47. The tungsten_skip_all Command
9.48. The tungsten_show_processlist Script
9.49. The tungsten_skip_seqno Script
9.50. The undeployall Command
9.51. The zabbix_tungsten_latency Command
9.52. The zabbix_tungsten_online Command
9.53. The zabbix_tungsten_progress Command
9.54. The zabbix_tungsten_services Command
10. The tpm Deployment Command
10.1. Comparing Staging and INI tpm Methods
10.2. Processing Installs and Upgrades
10.3. tpm Staging Configuration
10.3.1. Configuring default options for all services
10.3.2. Configuring a single service
10.3.3. Configuring a single host
10.3.4. Reviewing the current configuration
10.3.5. Installation
10.3.6. Upgrades from a Staging Directory
10.3.7. Configuration Changes from a Staging Directory
10.3.8. Converting from INI to Staging
10.4. tpm INI File Configuration
10.4.1. Creating an INI file
10.4.2. Installation with INI File
10.4.3. Upgrades with an INI File
10.4.4. Configuration Changes with an INI file
10.4.5. Converting from Staging to INI
10.4.6. Using the translatetoini.pl Script
10.5. tpm Commands
10.5.1. tpm ask Command
10.5.2. tpm cert Command
10.5.3. tpm check Command
10.5.4. tpm configure Command
10.5.5. tpm connector Command
10.5.6. tpm copy Command
10.5.7. tpm delete-service Command
10.5.8. tpm diag Command
10.5.9. tpm fetch Command
10.5.10. tpm firewall Command
10.5.11. tpm find-seqno Command
10.5.12. tpm generate-haproxy-for-api Command
10.5.13. tpm help Command
10.5.14. tpm install Command
10.5.15. tpm keep Command
10.5.16. tpm mysql Command
10.5.17. tpm policy Command
10.5.18. tpm post-process Command
10.5.19. tpm promote-connector Command
10.5.20. tpm purge-thl Command
10.5.21. tpm query Command
10.5.22. tpm report Command
10.5.23. tpm reset Command
10.5.24. tpm reset-thl Command
10.5.25. tpm reverse Command
10.5.26. tpm uninstall Command
10.5.27. tpm update Command
10.5.28. tpm validate Command
10.5.29. tpm validate-update Command
10.6. tpm Common Options
10.7. tpm Validation Checks
10.8. tpm Configuration Options
10.8.1. A tpm Options
10.8.2. B tpm Options
10.8.3. C tpm Options
10.8.4. D tpm Options
10.8.5. E tpm Options
10.8.6. F tpm Options
10.8.7. H tpm Options
10.8.8. I tpm Options
10.8.9. J tpm Options
10.8.10. L tpm Options
10.8.11. M tpm Options
10.8.12. N tpm Options
10.8.13. O tpm Options
10.8.14. P tpm Options
10.8.15. R tpm Options
10.8.16. S tpm Options
10.8.17. T tpm Options
10.8.18. U tpm Options
10.8.19. W tpm Options
11. Tungsten REST API (APIv2)
11.1. Getting Started with Tungsten REST API
11.1.1. Configuring the API
11.1.2. How to Access the API
11.1.3. Data Structures
11.2. Proxy (Connector) API Specifics
11.2.1. Proxy States
11.2.2. List and Set Proxy Configuration
11.2.3. User Map
11.2.4. Cluster Configuration Manipulation
11.3. Manager API Specifics
11.3.1. Manager Status
11.3.2. Cluster Topology
11.3.3. Service Status
11.3.4. Datasource Status
11.3.5. Cluster Control Commands
11.4. Replicator API Specifics
11.4.1. Replicator Endpoints
11.4.2. Service Endpoints
11.4.3. Service THL Endpoints
12. Replication Filters
12.1. Enabling/Disabling Filters
12.2. Enabling Additional Filters
12.3. Filter Status
12.4. Filter Reference
12.4.1. ansiquotes.js Filter
12.4.2. BidiRemoteSlave (BidiSlave) Filter
12.4.3. breadcrumbs.js Filter
12.4.4. CaseTransform Filter
12.4.5. ColumnName Filter
12.4.6. ConvertStringFromMySQL Filter
12.4.7. DatabaseTransform (dbtransform) Filter
12.4.8. dbrename.js Filter
12.4.9. dbselector.js Filter
12.4.10. dbupper.js Filter
12.4.11. dropcolumn.js Filter
12.4.12. dropcomments.js Filter
12.4.13. dropddl.js Filter
12.4.14. dropmetadata.js Filter
12.4.15. droprow.js Filter
12.4.16. dropstatementdata.js Filter
12.4.17. dropsqlmode.js Filter
12.4.18. dropxa.js Filter
12.4.19. Dummy Filter
12.4.20. EnumToString Filter
12.4.21. EventMetadata Filter
12.4.22. foreignkeychecks.js Filter
12.4.23. Heartbeat Filter
12.4.24. insertsonly.js Filter
12.4.25. Logging Filter
12.4.26. maskdata.js Filter
12.4.27. MySQLSessionSupport (mysqlsessions) Filter
12.4.28. mapcharset Filter
12.4.29. NetworkClient Filter
12.4.30. nocreatedbifnotexists.js Filter
12.4.31. OptimizeUpdates Filter
12.4.32. PrimaryKey Filter
12.4.33. PrintEvent Filter
12.4.34. Rename Filter
12.4.35. Replicate Filter
12.4.36. ReplicateColumns Filter
12.4.37. Row Add Database Name Filter
12.4.38. Row Add Transaction Info Filter
12.4.39. SetToString Filter
12.4.40. Shard Filter
12.4.41. shardbyrules.js Filter
12.4.42. shardbyseqno.js Filter
12.4.43. shardbytable.js Filter
12.4.44. SkipEventByType Filter
12.4.45. TimeDelay (delay) Filter
12.4.46. TimeDelayMsFilter (delayInMS) Filter
12.4.47. tosingledb.js Filter
12.4.48. truncatetext.js Filter
12.4.49. zerodate2null.js Filter
12.5. Standard JSON Filter Configuration
12.5.1. Rule Handling and Processing
12.5.2. Schema, Table, and Column Selection
12.6. JavaScript Filters
12.6.1. Writing JavaScript Filters
12.6.2. Installing Custom JavaScript Filters
13. Performance, Tuning and Testing
13.1. Block Commit
13.1.1. Monitoring Block Commit Status
13.2. Improving Network Performance
13.3. Tungsten Replicator Block Commit and Memory Usage
13.4. Connector Memory Management
13.5. Functional Testing
13.5.1. Manual and Automatic Failover
13.5.2. Backup and Restore
13.5.3. Connectivity
13.5.4. Performance and Other Tests
A. Release Notes
A.1. Tungsten Clustering 7.1.4 GA (1 Oct 2024)
A.2. Tungsten Clustering 7.1.3 GA (25 Jun 2024)
A.3. Tungsten Clustering 7.1.2 GA (3 Apr 2024)
A.4. Tungsten Clustering 7.1.1 GA (15 Dec 2023)
A.5. Tungsten Clustering 7.1.0 GA (16 Aug 2023)
B. Prerequisites
B.1. Staging Host Configuration
B.2. Host Configuration
B.2.1. Operating System Version Support
B.2.2. Creating the User Environment
B.2.3. Configuring Network and SSH Environment
B.2.4. Directory Locations and Configuration
B.2.5. Configure Software
B.2.6. sudo Configuration
B.2.7. SELinux Configuration
B.3. MySQL Database Setup
B.3.1. MySQL Version Support
B.3.2. MySQL Configuration
B.3.3. MySQL Configuration for Active/Active Deployments
B.3.4. MySQL Configuration for Heterogeneous Deployments
B.3.5. MySQL User Configuration
B.3.6. MySQL Unprivileged Users
B.4. Prerequisite Checklist
C. Troubleshooting
C.1. Contacting Support
C.1.1. Support Request Procedure
C.1.2. Creating a Support Account
C.1.3. Open a Support Ticket
C.1.4. Open a Support Ticket via Email
C.1.5. Getting Updates for all Company Support Tickets
C.1.6. Support Severity Level Definitions
C.2. Support Tools
C.2.1. Generating Diagnostic Information
C.2.2. Generating Advanced Diagnostic Information
C.2.3. Using tungsten_upgrade_manager
C.3. Error/Cause/Solution
C.3.1. Lots of entries added to replicator log
C.3.2. Backup/Restore is not bringing my host back to normal
C.3.3. Services requires a reset
C.3.4. Error: could not settle on encryption_client algorithm
C.3.5. ERROR backup.BackupTask Backup operation failed: null
C.3.6. Unable to update the configuration of an installed directory
C.3.7. Cluster remains in MAINTENANCE mode after tpm update
C.3.8. Missing events, or events not extracted correctly
C.3.9. Triggers not firing correctly on Replica
C.3.10. Replicator reports an Out of Memory error
C.3.11. [S1000][unixODBC][MySQL][ODBC 5.3(w) Driver]SSL connection error: unknown error number [ISQL]ERROR: Could not SQLConnect
C.3.12. Latency is high: master:ONLINE, progress=41331580333, THL latency=78849.733
C.3.13. Connector shows errors with "java.net.SocketException: Broken pipe"
C.3.14. The Primary replicator stopped with a JDBC error.
C.3.15. cctrl reports MANAGER(state=STOPPED)
C.3.16. trepctl status hangs
C.3.17. Attempt to write new log record with equal or lower fragno: seqno=3 previous stored fragno=32767 attempted new fragno=-32768
C.3.18. Replicator runs out of memory
C.3.19. ERROR 1010 (HY000) at line 5094506: Error dropping database (can't rmdir './mysql-bin/', errno: 17)
C.3.20. ERROR >> host1 >> can't alloc thread
C.3.21. ERROR 1580 (HY000) at line 5093787: You cannot 'DROP' a log table if logging is enabled
C.3.22. WARNING: An illegal reflective access operation has occurred
C.3.23. ERROR 2013 (HY000) at line 583: Lost connection to MySQL server during query
C.3.24. pendingExceptionMessage": "Unable to update last commit seqno: Incorrect datetime value: '2016-03-13 02:02:26' for column 'update_timestamp' at row 1
C.3.25. Too many open processes or files
C.3.26. Replication latency very high
C.3.27. Backup agent name not found: xtrabackup-full
C.3.28. WARN [KeepAliveTimerTask] - Error while sending a KEEP_ALIVE query to connection.
C.3.29. Event application failed: seqno=20725782 fragno=0 message=java.sql.SQLDataException: Data too long for column 'eventid' at row 1
C.3.30. MySQL 8.0+, User Roles and Smartscale
C.3.31. element 'mysql_readonly' not found in path
C.3.32. cctrl hangs
C.3.33. Replicator fails to connect after updating password
C.3.34. There were issues configuring the sandbox MySQL server
C.3.35. Starting replication after performing a restore because of an invalid restart sequence number
C.3.36. MySQL is incorrectly configured
C.4. Known Issues
C.4.1. Triggers
C.5. Troubleshooting Timeouts
C.6. Troubleshooting Backups
C.7. Running Out of Diskspace
C.8. Troubleshooting SSH and tpm
C.9. Troubleshooting Data Differences
C.9.1. Identify Structural Differences
C.9.2. Identify Data Differences
C.10. Comparing Table Data
C.11. Troubleshooting Memory Usage
D. Files, Directories, and Environment
D.1. The Tungsten Cluster Install Directory
D.1.1. The backups Directory
D.1.2. The releases Directory
D.1.3. The service_logs Directory
D.1.4. The share Directory
D.1.5. The thl Directory
D.1.6. The tungsten Directory
D.2. Log Files
D.3. Environment Variables
E. Terminology Reference
E.1. Transaction History Log (THL)
E.1.1. THL Format
E.2. Generated Field Reference
E.2.1. Terminology: Fields masterConnectUri
E.2.2. Terminology: Fields masterListenUri
E.2.3. Terminology: Fields accessFailures
E.2.4. Terminology: Fields active
E.2.5. Terminology: Fields activeSeqno
E.2.6. Terminology: Fields appliedLastEventId
E.2.7. Terminology: Fields appliedLastSeqno
E.2.8. Terminology: Fields appliedLatency
E.2.9. Terminology: Fields applier.class
E.2.10. Terminology: Fields applier.name
E.2.11. Terminology: Fields applyTime
E.2.12. Terminology: Fields autoRecoveryEnabled
E.2.13. Terminology: Fields autoRecoveryTotal
E.2.14. Terminology: Fields averageBlockSize
E.2.15. Terminology: Fields blockCommitRowCount
E.2.16. Terminology: Fields cancelled
E.2.17. Terminology: Fields channel
E.2.18. Terminology: Fields channels
E.2.19. Terminology: Fields clusterName
E.2.20. Terminology: Fields commits
E.2.21. Terminology: Fields committedMinSeqno
E.2.22. Terminology: Fields criticalPartition
E.2.23. Terminology: Fields currentBlockSize
E.2.24. Terminology: Fields currentEventId
E.2.25. Terminology: Fields currentLastEventId
E.2.26. Terminology: Fields currentLastFragno
E.2.27. Terminology: Fields currentLastSeqno
E.2.28. Terminology: Fields currentTimeMillis
E.2.29. Terminology: Fields dataServerHost
E.2.30. Terminology: Fields discardCount
E.2.31. Terminology: Fields doChecksum
E.2.32. Terminology: Fields estimatedOfflineInterval
E.2.33. Terminology: Fields eventCount
E.2.34. Terminology: Fields extensions
E.2.35. Terminology: Fields extractTime
E.2.36. Terminology: Fields extractor.class
E.2.37. Terminology: Fields extractor.name
E.2.38. Terminology: Fields filter.#.class
E.2.39. Terminology: Fields filter.#.name
E.2.40. Terminology: Fields filterTime
E.2.41. Terminology: Fields flushIntervalMillis
E.2.42. Terminology: Fields fsyncOnFlush
E.2.43. Terminology: Fields headSeqno
E.2.44. Terminology: Fields intervalGuard
E.2.45. Terminology: Fields lastCommittedBlockSize
E.2.46. Terminology: Fields lastCommittedBlockTime
E.2.47. Terminology: Fields latestEpochNumber
E.2.48. Terminology: Fields logConnectionTimeout
E.2.49. Terminology: Fields logDir
E.2.50. Terminology: Fields logFileRetainMillis
E.2.51. Terminology: Fields logFileSize
E.2.52. Terminology: Fields maxChannel
E.2.53. Terminology: Fields maxDelayInterval
E.2.54. Terminology: Fields maxOfflineInterval
E.2.55. Terminology: Fields maxSize
E.2.56. Terminology: Fields maximumStoredSeqNo
E.2.57. Terminology: Fields minimumStoredSeqNo
E.2.58. Terminology: Fields name
E.2.59. Terminology: Fields offlineRequests
E.2.60. Terminology: Fields otherTime
E.2.61. Terminology: Fields pendingError
E.2.62. Terminology: Fields pendingErrorCode
E.2.63. Terminology: Fields pendingErrorEventId
E.2.64. Terminology: Fields pendingErrorSeqno
E.2.65. Terminology: Fields pendingExceptionMessage
E.2.66. Terminology: Fields pipelineSource
E.2.67. Terminology: Fields processedMinSeqno
E.2.68. Terminology: Fields queues
E.2.69. Terminology: Fields readOnly
E.2.70. Terminology: Fields relativeLatency
E.2.71. Terminology: Fields resourcePrecedence
E.2.72. Terminology: Fields rmiPort
E.2.73. Terminology: Fields role
E.2.74. Terminology: Fields seqnoType
E.2.75. Terminology: Fields serializationCount
E.2.76. Terminology: Fields serialized
E.2.77. Terminology: Fields serviceName
E.2.78. Terminology: Fields serviceType
E.2.79. Terminology: Fields shard_id
E.2.80. Terminology: Fields simpleServiceName
E.2.81. Terminology: Fields siteName
E.2.82. Terminology: Fields sourceId
E.2.83. Terminology: Fields stage
E.2.84. Terminology: Fields started
E.2.85. Terminology: Fields state
E.2.86. Terminology: Fields stopRequested
E.2.87. Terminology: Fields store.#
E.2.88. Terminology: Fields storeClass
E.2.89. Terminology: Fields syncInterval
E.2.90. Terminology: Fields taskCount
E.2.91. Terminology: Fields taskId
E.2.92. Terminology: Fields timeInCurrentEvent
E.2.93. Terminology: Fields timeInStateSeconds
E.2.94. Terminology: Fields timeoutMillis
E.2.95. Terminology: Fields totalAssignments
E.2.96. Terminology: Fields transitioningTo
E.2.97. Terminology: Fields uptimeSeconds
E.2.98. Terminology: Fields version
F. Internals
F.1. Extending Backup and Restore Behavior
F.1.1. Backup Behavior
F.1.2. Restore Behavior
F.1.3. Writing a Custom Backup/Restore Script
F.1.4. Enabling a Custom Backup Script
F.2. Character Sets in Database and Tungsten Cluster
F.3. Understanding Replication of Date/Time Values
F.3. Best Practices
F.4. Memory Tuning and Performance
F.4.1. Understanding Tungsten Replicator Memory Tuning
F.4.2. Connector Memory Management
F.5. Tungsten Replicator Pipelines and Stages
F.6. Tungsten Cluster Schemas
G. Frequently Asked Questions (FAQ)
G.1. General Questions
G.2. Cloud Deployment and Management
H. Ecosystem Support
H.1. Continuent Github Repositories
I. Configuration Property Reference

List of Figures

3.1. Topologies: Standalone HA Cluster
3.2. Topologies: Composite Active/Passive Cluster
3.3. Topologies: Multi-Site/Active-Active Clusters
3.4. Topologies: Composite Active/Active Clusters
3.5. Topologies: Composite Dynamic Active/Active Clusters
3.6. Topologies: Replicating into a Dataservice
3.7. Topologies: Replicating Data Out of a Cluster
3.8. Topologies: Replication from a Cluster to an Offboard Datawarehouse
3.9. Topologies: Replication from a Cluster to an Offboard Datawarehouse
3.10. Migration: Migrating Native Replication using a New Service
5.1. Security Internals: Cluster Communication Channels
6.1. Migration: Migrating Native Replication using a New Service
6.2. Cacti Monitoring: Example Graphs
7.1. Tungsten Connector Basic Architecture
7.2. Basic MySQL/Application Connectivity
7.3. Advanced MySQL/Application Connectivity
7.4. Using Tungsten Connector for MySQL/Application Connectivity
7.5. Tungsten Connector during a failed datasource
7.6. Tungsten Connector routing architecture
7.7. Tungsten Connector Bridge Mode Architecture
7.8. Tungsten Connector Authentication
8.1. Failover Scenario 1
8.2. Failover Scenario 2
8.3. Failover Scenario 3
10.1. tpm Staging Based Deployment
10.2. tpm INI Based Deployment
10.3. Internals: Cluster Communication Channels
12.1. Filters: Pipeline Stages on Extractors
12.2. Filters: Pipeline Stages on Appliers
B.1. Tungsten Deployment

List of Tables

1.1. Key Terminology
2.1. Key Terminology
2.2. Tungsten OS Support
2.3. MySQL/Tungsten Version Support
7.1. Routing Method Selection
7.2. Connector Command Line Sub-Commands
7.3. Inline Interface Commands
9.1. cctrl Command-line Options
9.2. cctrl Command-line Options
9.3. cctrl Command-line Options
9.4. cctrl Command-line Options
9.5. cctrl Command-line Options
9.6. cctrl Command-line Options
9.7. cctrl Command-line Options
9.8. cctrl Command-line Options
9.9. cctrl Command-line Options
9.10. cctrl Command-line Options
9.11. cctrl Command-line Options
9.12. cctrl Commands
9.13. cctrldatasource Commands
9.14. check_tungsten_latency Options
9.15. check_tungsten_online Options
9.16. check_tungsten_policy Options
9.17. check_tungsten_progress Options
9.18. check_tungsten_services Options
9.19. cluster_backup Command-line Options
9.20. connector Commands
9.21. ddlscan Command-line Options
9.22. ddlscan Supported Templates
9.23. dsctl Commands
9.24. dsctl Command-line Options
9.25. dsctl Command-line Options
9.26. dsctl Command-line Options
9.27. manager Commands
9.28. multi_trepctl Command-line Options
9.29. multi_trepctl--output Option
9.30. multi_trepctl Commands
9.31. query Common Options
9.32. replicator Commands
9.33. replicator Commands Options for condrestart
9.34. replicator Commands Options for console
9.35. replicator Commands Options for restart
9.36. replicator Commands Options for start
9.37. tapi Generic Options
9.38. tapi CURL-related Options
9.39. tapi Nagios/NRPE/Zabbix-related Options
9.40. tapi Admin-related Options
9.41. tapi Filter-related Options
9.42. tapi API-related Options
9.43. tapi Status-related Options
9.44. tapi Backup and Restore-related Options
9.45. thl Options
9.46. trepctl Command-line Options
9.47. trepctl Replicator Wide Commands
9.48. trepctl Service Commands
9.49. trepctl backup Command Options
9.50. trepctl clients Command Options
9.51. trepctl offline-deferred Command Options
9.52. trepctl online Command Options
9.53. trepctl pause Command Options
9.54. trepctl purge Command Options
9.55. trepctl reset Command Options
9.56. trepctl resume Command Options
9.57. trepctl setdynamic Command Options
9.58. trepctl setrole Command Options
9.59. trepctl shard Command Options
9.60. trepctl status Command Options
9.61. trepctl wait Command Options
9.62. tmonitor Common Options
9.63. tpasswd Common Options
9.64. tprovision Command-line Options
9.65. tungsten_find_orphaned Options
9.66. tungsten_find_position Options
9.67. tungsten_find_seqno Options
9.68. tungsten_get_mysql_datadir Command-line Options
9.69. tungsten_get_ports Options
9.70. tungsten_health_check Command-line Options
9.71. tungsten_merge_logs Command-line Options
9.72. tungsten_monitor Command-line Options
9.73. tungsten_monitor Command-line Options
9.74. tungsten_post_process Options
9.75. tungsten_prep_upgrade Command-line Options
9.76. tungsten_purge_thl Options
9.77. tungsten_reset_manager Options
9.78. tungsten_send_diag Command-line Options
9.79. tungsten_skip_all Options
9.80. tungsten_show_processlist Command-line Options
9.81. tungsten_skip_seqno Command-line Options
9.82. zabbix_tungsten_latency Options
9.83. zabbix_tungsten_online Options
9.84. zabbix_tungsten_progress Options
9.85. check_tungsten_services Options
10.1. TPM Deployment Methods
10.2. tpm Core Options
10.3. tpm Commands
10.4. tpm ask Common Options
10.5. tpm cert Read-Only Actions
10.6. tpm cert Write Actions
10.7. tpm cert Arguments
10.8. Convenience tags
10.9. typeSpecs for tpm cert ask
10.10. typeSpecs for tpm cert example
10.11. typeSpecs for tpm cert gen
10.12. typeSpecs for tpm cert vi
10.13. Options for tpm check
10.14. Options for tpm check ini
10.15. Options for tpm check ports
10.16. tpm copy Common Options
10.17. tpm delete-service Common Options
10.18. tpm find-seqno Options
10.19. tpm generate-haproxy-for-api Common Options
10.20. tpm keep Options
10.21. tpm policy Options
10.22. tpm post-process Options
10.23. tpm purge-thl Options
10.24. tpm report Common Options
10.25. tpm Common Options
10.26. tpm Validation Checks
10.27. tpm Configuration Options
C.1. tungsten_upgrade_manager Options
D.1. Continuent Tungsten Directory Structure
D.2. Continuent Tungsten tungsten Sub-Directory Structure
E.1. THL Event Format
   Next
   Preface
Copyright © 2016-2023 Continuent, Ltd.(800) 270-9035Terms of UsePrivacy & CookiesLegal NoticeExhibits