2.28. Tungsten Replicator 6.1.4 GA (4 June 2020)

Version End of Life. 15 Aug 2024

Release 6.1.4 contains a number of improvements and bug fixes, specifically for the tpm command line tool and improvements to the Redshift Applier. In addition this release now fully supports the latest binlog compression feature of MySQL 8.0.20.

Improvements, new features and functionality

  • Command-line Tools

    • Improves tpm performance by using more efficient routines to calculate paths.

      Issues: CT-1168

    • Added the ability for tpm diag to skip both individual gather subroutines along with entire groups of gather subroutines.

      Also added ability to list all gather groups and subroutines using --list for use with the --skip and --skipgroups cli arguments.

      Issues: CT-1176

    • tungsten_provision_slave has been rewritten fixing a number of issues in the previous release. This version was previously released as the Beta script prov-sl.sh.

      Issues: CT-1208

    • A number of performance improvements and fixes have been incorporated into the Drizzle Driver used for communication between components and MySQL, these include:

      • Better handling of large queries close to max network packet size.

      • Batch Support. Instead of sending statements one by one, the driver will be able to send multiple statements at once, avoiding round trips between the driver and MySQL server.

      • Fixes issues with interpreting useSSL on connect string URLs.

      Issues: CT-1215, CT-1216, CT-1217, CT-1228

    • The tungsten_send_diag command now accepts arguments for the tpm diag command using --args or -a for short.

      You must enclose the arguments in quotes, for example:

      shell> tungsten_send_diag -c 9999 -d --args ‘--all -v’

      Issues: CT-1220

  • Core Replicator

    • debug has been disabled by default in the schemachange filter. Resulting in reduced noise in the replicator log file.

      Issues: CT-1013

    • A new replicator role thl-client has been added. This new role allows the replicator to download THL from a Primary, but not apply to the target database.

      Issues: CT-1123

    • A new delayInMs filter has been added which allows the applying of THL to a Replica to be delayed. The filter allows millisecond precision. This filter works in the same way as the TimeDelayFilter, however that filter only allow second precision.

      Issues: CT-1191

    • A new droprow JavaScript filter has been added.

      The filter works at ROW level and allows the filtering out of rows based on one or more column/value matches

      Issues: CT-1213

    • When configuring the Redshift applier, you can now configure which tool the applier will use for posting CSV files to S3. Options are s3cmd (default), s4cmd or aws.

      Issues: CT-1218

    • A number of improvements have been made to the Redshift Applier to allow optional levels of table locking.

      This is particularly useful when you have multiple Redshift Appliers in a Fan-In topology, and/or very high volumes of data to process.

      The additional locking options reduce the risk of Redshift Serializable Isolation Violation errors occuring.

      Full details of how to utilise the new options can be read at Handling Concurrent Writes from Multiple Appliers

      Issues: CT-1221

    • Tungsten Replicator can now correctly extract and parse Binary Log entries when the MySQL option binlog-transaction-compression has been enabled.

      binlog-transaction-compression is a new parameter introduced from MySQL 8.0.20.

      Issues: CT-1223

Bug Fixes

  • Command-line Tools

    • In certain edge cases, tungsten_provision_slave would fail to detect if mysql was shutdown.

      Issues: CT-1096

    • tpm diag now collects directories specified with !includedir in the /etc/my.cnf file.

      Issues: CT-1153

    • Fixes the tpm update command, which would exit with the error:

      Argument " (error code 1)" isn't numeric

      Issues: CT-1165

    • tpm diag now collects any files specified by !include directives in the /etc/my.cnf and /etc/mysql/my.cnf files.

      tpm diag also looks in /etc/mysql/my.cnf for !includedir directives

      Issues: CT-1169

    • Fixes a bug which prevented tungsten_send_diag from uploading a self-generated diagnostic zip file.

      Issues: CT-1170

    • tpm diag now properly derives the correct target path to the releases directory if the home directory in the configuration points to a sym-link.

      Issues: CT-1172

    • Removed tpm diag call to sudo for gathering ifconfig and lsb_release commands.

      Issues: CT-1175

    • tpm update would fail and report errors if deployall had been executed.

      Issues: CT-1179

    • tpm diag no longer requires the mysql command-line client when running on non-MySQL Applier nodes, and no longer attempts to gather any MySQL diagnostic information.

      Issues: CT-1188

    • Removes the requirement to execute component start/stop commands with sudo when called through systemd.

      This is specific to start/stop actions following the use of the deployall scripts.

      Issues: CT-1193

    • Fixes cases where tpm fails when the OS hostname command returns a different string than what is used in the configuration (i.e. hostname returns a FQDN, yet the configuration contains shortnames like db1, db2, etc.).

      Issues: CT-1206

    • In certain cases, after a reprovision, tungsten_provision_slave didn’t always run the steps to reset the local replicator service. This made the replicator go into an error state after provision had completed.

      Issues: CT-1210

    • The tpm diag command now handles the cluster-slave topology more gracefully, and properly handles cluster nodes without the Connector installed.

      Improved output text clarity by converting multiple verbose-level outputs to debug, and warnings to notice-level.

      Issues: CT-1222

    • tpm diag now gathers sym-linked files correctly.

      Issues: CT-1232

    • ddlscan now sets the datatype for sequence number columns to BIGINT when generating staging table DDL for Redshift deployments.

      Issues: CT-1235

    • Fixes a situation where tpm update exits with a Data::Dumper error.

      Issues: CT-1249

  • Core Replicator

    • In heterogeneous replicator deployments, the convertstringfrommysql filter would fail to convert strings for alphanumeric key values.

      Issues: CT-1128

    • Tungsten Replicator could wrongly think it is already in DEGRADED mode when trying to put it into DEGRADED state.

      Issues: CT-1131

    • Tungsten Replicator now recognises Amazon AWS SSL Certificates to enable SSL communication with AWS Aurora.

      Issues: CT-1173

    • When host server time (and thus MySQL time) is not configured as UTC, issuing cluster heartbeat or trepctl heartbeat in the first hours around daylight savings time would create an invalid time in MySQL.

      For more information on timezones when issuing heartbeats, see trepctl heartbeat Time Zone Handling

      Issues: CT-1174

    • The replicator would fail to apply into Vertica when configured as an offboard installation due to the applier incorrectly expecting the csv files to exist locally on the remote Vertica host.

      Issues: CT-1194

    • Due to a change in the Binary log structure introduced in MySQL 8.0.16, the replicator would fail to extract transactions for Partitioned tables.

      Issues: CT-1201

    • The replicator would fail to correctly parse statements with leading comment blocks in excess of 200 characters.

      Issues: CT-1236