Tungsten Replicator 7.1.0

Build: 182

Release Date: 16 Aug 2023

End of Life Date: 7 Jul 2026

Release 7.1.0 is the next major v7 release containing a number of important bug fixes and key new features.

Warning

Due to JGroup libraries being updated in this release, managers running releases older than 7.1.0 will not communicate with managers running 7.1.0+ therefore when upgrading to this release, all nodes must be upgraded before proper cluster communication will be restored. Ensure the cluster is in MAINTENANCE before beginning the upgrade and do NOT SHUN nodes whilst a mix of manager versions are running.

Behavior Changes (2)

The following changes may affect existing scripts and integration tools. Any scripts or environment which make use of these tools should check and update for the new configuration:

Command-line Tools (1)

• The tpm copy-keys command has been renamed to tpm copy and a new command has been createdtpm cert copy with the same functionality.
  Issue: CT-2186

Backup and Restore (1)

• When performing a provision via rsync, tprovision will now sleep for 2 seconds after locking tables to make sure all transactions have finished writing to disk.
  Issue: CT-2169

Improvements, new features and functionality (16)

Installation and Deployment (2)

• Added tpm flag deploy-systemd as a more meaningful alias to install
  Issue: CT-2152
• Running tpm uninstall will now save all of the Tungsten database tracking schemas for later use. There is also a new tpm keep command, which allows the tracking schemas to be saved to disk at any time in multiple formats (.json, .dmp and .cmd)
  Issue: CT-2131

Command-line Tools (5)

• The tungsten_send_diag command has a new argument --all which will tell thetpm diag command to gather all hosts with -a, and this replaces the previous method for gathering all hosts for tungsten_send_diag, --args '--all'
  Issue: CT-2150
• The tpm diag command now supports the --skipsudo and --nosudoarguments to prevent operations from using the sudo command. Using this option may result in tpm diagskipping/failing various gathers due to a lack of access.
  Issue: CT-2146

• The tpm report command now displays the security-specific information for each channel shown, including file paths and tpm options. Security information and tpm options for each channel will also be shown when --extra is used with --list.

  Also added the tpm ask certs command with expiry and sha2566 info per alias, along with thetpm ask certtpm and tpm ask certlocations commands with reference information about the security.properties file.

  Added help text for tpm ask --long which shows the key/variable name along with the value. The tmonitor -t test command now shows only the actual Tungsten-specific metrics lines.

  Added tmonitor -T test to show metrics help and headers along with actual tungsten_* metrics lines

  Issue: CT-2088

• A new command tpm cert has been added to aid in the creation, rotation and management of certificates for all areas of Tungsten.

  Known limitations: Percona 5.6, and all versions of MariaDB do not provide the mysql_ssl_rsa_setup command required bytpm cert gen mysqlcerts.

  Issue: CT-2085
• Added a new option --preserve-schema to the tpm uninstall command in order to leave the tracking schema in the database.
  Issue: CT-561

Backup and Restore (3)

• The tprovision script will sleep for 5 seconds by default when using the rsync method after issuing a flush logs. The sleep value is configurable as a command line option --flush-after-sleep.
  Issue: CT-2101
• tungsten_get_mysql_datadir can now return additional mysql database directories.
  Issue: CT-1985
• MySQL clone can now be used as an option for recovery using tprovision.
  Issue: CT-1417

PostgreSQL Replication (2)

• Added a new ddlscan template, ddl-mysql-postgres-staging.vm, to allow generation of staging table DDL for the PostgreSQL batch applier.
  Issue: CT-2136
• It is now possible to configure the PostgreSQL applier in Batch mode.
  Issue: CT-2031

Amazon S3 Replication (1)

• A new applier is now available that replicates to csv files on S3 storage. For more information, see "Deploying the Amazon S3 Applier"
  Issue: CT-606

Filters (1)

• A new shardbyrules filter has been added that will allow rule-based sharding of replication based on user configurable rules that would allow sharding at table level, whereas previously sharding would only be handled at schema level.
  Issue: CT-2164

Monitoring (2)

• Prometheus libraries have been upgraded from version 0.8.1 to 0.16.0
  Issue: CT-2166
• Prometheus exporters now provide the ssl cert expiration date as an epoch value in addition to the label.
  Issue: CT-2099

Bug Fixes (12)

Installation and Deployment (1)

• Fixed RPM package script to run tpm install instead of tpm update when installing the rpm
  Issue: CT-2130

Command-line Tools (2)

• The tpm command now searches more places to locate shell commands that are called, especially useful when $CONTINUENT_ROOT/share/env.sh is not sourced.
  Issue: CT-2182
• tpm uninstall would sometimes print "ERROR >> db1 >> undefined method '+' for nil:NilClass"
  Issue: CT-2104

Backup and Restore (5)

• Using mysqldump for tprovision could incorrectly create a new SSL key pair/
  Issue: CT-2142
• tprovision will now timeout if ssh is blocked from the target to the source host.
  Issue: CT-2139
• tprovision would accept bogus options and not produce an error. This has now been fixed.
  Issue: CT-2045
• tprovision will now print an error message and exit if the MySQL datadir does not exist.
  Issue: CT-1901
• tprovision would produce errors if the local hostname were different from the hostname used in the Tungsten install (short vs long names).
  Issue: CT-1363

Core Replicator (2)

• Fixed an issue while processing geometry data with SRID 4326 that would swap longitude and latitude. This applies only to MySQL 8, as prior MySQL versions do not allow specifying the order when applying a WKB (Well-known binary) to MySQL.
  Issue: CT-2172

• Improved a query that is run by Tungsten when fetching tables metadata (column names, datatypes, etc). While it is not generally needed, the unoptimized query can run badly (especially) against old mysql versions with a lot of databases / tables. For now, the new optimized query is not used by default, but this could change in some future version.

  This can be enabled by using the following property :

  property=replicator.datasource.global.connectionSpec.usingOptimizedMetadataQuery=true

  Issue: CT-2077

PostgreSQL Replication (2)

• Fixed a NullPointerException error occurring at startup when applying to PostgreSQL.
  Issue: CT-2021
• Fixes an issue with the ddl-mysql-postgres.vm ddlscan template that caused varchar datatype lengths to be parsed incorrectly.
  Issue: CT-2019