1.3. Tungsten Clustering 6.1.4 GA (4 June 2020)

Version End of Life. Not Yet Set

Release 6.1.4 contains a number of improvements and bug fixes, specifically for the tpm command line tool and stability improvements for Composite Active/Active topologies. In addition this release now fully supports the latest binlog compression feature of MySQL 8.0.20.

Improvements, new features and functionality

  • Tungsten Manager

    • When a Primary is OFFLINE or SHUNNED, we no longer mark the whole data service as OFFLINE.

      Note

      This will allow reading from Replicas to continue even if the Primary is offline.

      Issues: CT-1152

    • In a Composite Active/Active topology, the relay hosts, by default, will pull THL from the remote Primaries. This can be changed to pull from remote Replica(s) by use of the following setting policy-relay-from-slave=true (in [Tungsten Clustering (for MySQL) 6.1 Manual])

      Note

      This is a change in behaviour. From version 6.0.4 up to and including 6.1.3, the default behaviour was configured to pull from the remote Replica(s).

      Issues: CT-1211

Bug Fixes

  • Command-line Tools

    • The tpm diag (in [Tungsten Clustering (for MySQL) 6.1 Manual]) command now captures cctrl ls output from the Composite dataservice when appropriate.

      Issues: CT-1139

    • Fixes an issue where affinity specified in the configuration (connector-affinity (in [Tungsten Clustering (for MySQL) 6.1 Manual])) was ignored since it was not written to user.map (in [Tungsten Clustering (for MySQL) 6.1 Manual]) (this affects proxy configurations only).

      Issues: CT-1146

    • Fixed a bug where tpm diag (in [Tungsten Clustering (for MySQL) 6.1 Manual]) would fail to gather some MySQL information on a Composite Active/Active node.

      Issues: CT-1167

    • Fixed a bug in tungsten_post_process (in [Tungsten Clustering (for MySQL) 6.1 Manual]) where the /etc/tungsten/tungsten.ini (in [Tungsten Clustering (for MySQL) 6.1 Manual]) file would not be read properly. Also adds two warnings for skipped entries and corrects a typo in the property definitions.

      Issues: CT-1198

  • Tungsten Connector

    • Fixed an issue where connector client-list (in [Tungsten Clustering (for MySQL) 6.1 Manual]) and the Proxy-mode command, tungsten show processlist, would report NullPointerException errors when listing disconnected client applications.

      Issues: CT-1177

    • When in proxy mode and fetching data source configuration at startup, the Connector will now properly parse, and use, affinity (and other URL options if any) to establish the connection.

      Issues: CT-1212

    • Fixes an issue where connections with multiple affinity would not keep the current channel and reconnect to the database server while the connection could be reused.

      Issues: CT-1250

  • Tungsten Manager

    • In certain edge case situations, the manager could choose a manually shunned node as a viable node during failover.

      Issues: CT-1115

    • Issuing cluster topology validate within cctrl (in [Tungsten Clustering (for MySQL) 6.1 Manual]) would fail if the cluster contained an Active Witness host.

      Issues: CT-1180

    • Piping multiple commands to cctrl (in [Tungsten Clustering (for MySQL) 6.1 Manual]) that would affect components in a remote cluster would fail, for example:

      shell> echo "use east; replicator db5 offline" | cctrl

      Issues: CT-1209

    • Resolves an edge case in a Composite Active/Passive topology, with 2 or more Composite Passive dataservices, where a switch of a relay node in a single Replica service would incorrectly reconfigure all relays in the other Replica services.

      Issues: CT-1214

Tungsten Clustering 6.1.4 Includes the following changes made in Tungsten Replicator 6.1.4

Release 6.1.4 contains a number of improvements and bug fixes, specifically for the tpm command line tool and improvements to the Redshift Applier. In addition this release now fully supports the latest binlog compression feature of MySQL 8.0.20.

Improvements, new features and functionality

  • Command-line Tools

    • Improves tpm (in [Tungsten Replicator 6.1 Manual]) performance by using more efficient routines to calculate paths.

      Issues: CT-1168

    • Added the ability for tpm diag (in [Tungsten Replicator 6.1 Manual]) to skip both individual gather subroutines along with entire groups of gather subroutines.

      Also added ability to list all gather groups and subroutines using --list for use with the --skip and --skipgroups cli arguments.

      Issues: CT-1176

    • tungsten_provision_slave (in [Tungsten Replicator 6.1 Manual]) has been rewritten fixing a number of issues in the previous release. This version was previously released as the Beta script prov-sl.sh.

      Issues: CT-1208

    • A number of performance improvements and fixes have been incorporated into the Drizzle Driver used for communication between components and MySQL, these include:

      • Better handling of large queries close to max network packet size.

      • Batch Support. Instead of sending statements one by one, the driver will be able to send multiple statements at once, avoiding round trips between the driver and MySQL server.

      • Fixes issues with interpreting useSSL on connect string URLs.

      Issues: CT-1215, CT-1216, CT-1217, CT-1228

    • The tungsten_send_diag (in [Tungsten Replicator 6.1 Manual]) command now accepts arguments for the tpm diag (in [Tungsten Replicator 6.1 Manual]) command using --args or -a for short.

      You must enclose the arguments in quotes, for example:

      shell> tungsten_send_diag -c 9999 -d --args ‘--all -v’

      Issues: CT-1220

  • Core Replicator

    • debug has been disabled by default in the schemachange filter. Resulting in reduced noise in the replicator log file.

      Issues: CT-1013

    • A new replicator role thl-client has been added. This new role allows the replicator to download THL from a Primary, but not apply to the target database.

      Issues: CT-1123

    • A new delayInMs (in [Tungsten Replicator 6.1 Manual]) filter has been added which allows the applying of THL to a Replica to be delayed. The filter allows millisecond precision. This filter works in the same way as the TimeDelayFilter (in [Tungsten Replicator 6.1 Manual]), however that filter only allow second precision.

      Issues: CT-1191

    • A new droprow JavaScript filter has been added.

      The filter works at ROW level and allows the filtering out of rows based on one or more column/value matches

      Issues: CT-1213

    • When configuring the Redshift applier, you can now configure which tool the applier will use for posting CSV files to S3. Options are s3cmd (default), s4cmd or aws.

      Issues: CT-1218

    • A number of improvements have been made to the Redshift Applier to allow optional levels of table locking.

      This is particularly useful when you have multiple Redshift Appliers in a Fan-In topology, and/or very high volumes of data to process.

      The additional locking options reduce the risk of Redshift Serializable Isolation Violation errors occuring.

      Full details of how to utilise the new options can be read at Handling Concurrent Writes from Multiple Appliers (in [Tungsten Replicator 6.1 Manual])

      Issues: CT-1221

    • Tungsten Replicator can now correctly extract and parse Binary Log entries when the MySQL option binlog-transaction-compression has been enabled.

      binlog-transaction-compression is a new parameter introduced from MySQL 8.0.20.

      Issues: CT-1223

Bug Fixes

  • Command-line Tools

    • In certain edge cases, tungsten_provision_slave (in [Tungsten Replicator 6.1 Manual]) would fail to detect if mysql was shutdown.

      Issues: CT-1096

    • tpm diag (in [Tungsten Replicator 6.1 Manual]) now collects directories specified with !includedir in the /etc/my.cnf file.

      Issues: CT-1153

    • Fixes the tpm update (in [Tungsten Replicator 6.1 Manual]) command, which would exit with the error:

      Argument " (error code 1)" isn't numeric

      Issues: CT-1165

    • tpm diag (in [Tungsten Replicator 6.1 Manual]) now collects any files specified by !include directives in the /etc/my.cnf and /etc/mysql/my.cnf files.

      tpm diag (in [Tungsten Replicator 6.1 Manual]) also looks in /etc/mysql/my.cnf for !includedir directives

      Issues: CT-1169

    • Fixes a bug which prevented tungsten_send_diag (in [Tungsten Replicator 6.1 Manual]) from uploading a self-generated diagnostic zip file.

      Issues: CT-1170

    • tpm diag (in [Tungsten Replicator 6.1 Manual]) now properly derives the correct target path to the releases directory if the home directory in the configuration points to a sym-link.

      Issues: CT-1172

    • Removed tpm diag (in [Tungsten Replicator 6.1 Manual]) call to sudo for gathering ifconfig and lsb_release commands.

      Issues: CT-1175

    • tpm update (in [Tungsten Replicator 6.1 Manual]) would fail and report errors if deployall (in [Tungsten Replicator 6.1 Manual]) had been executed.

      Issues: CT-1179

    • tpm diag (in [Tungsten Replicator 6.1 Manual]) no longer requires the mysql command-line client when running on non-MySQL Applier nodes, and no longer attempts to gather any MySQL diagnostic information.

      Issues: CT-1188

    • Removes the requirement to execute component start/stop commands with sudo when called through systemd.

      This is specific to start/stop actions following the use of the deployall (in [Tungsten Replicator 6.1 Manual]) scripts.

      Issues: CT-1193

    • Fixes cases where tpm (in [Tungsten Replicator 6.1 Manual]) fails when the OS hostname command returns a different string than what is used in the configuration (i.e. hostname returns a FQDN, yet the configuration contains shortnames like db1, db2, etc.).

      Issues: CT-1206

    • In certain cases, after a reprovision, tungsten_provision_slave (in [Tungsten Replicator 6.1 Manual]) didn’t always run the steps to reset the local replicator service. This made the replicator go into an error state after provision had completed.

      Issues: CT-1210

    • The tpm diag (in [Tungsten Replicator 6.1 Manual]) command now handles the cluster-slave topology more gracefully, and properly handles cluster nodes without the Connector installed.

      Improved output text clarity by converting multiple verbose-level outputs to debug, and warnings to notice-level.

      Issues: CT-1222

    • tpm diag (in [Tungsten Replicator 6.1 Manual]) now gathers sym-linked files correctly.

      Issues: CT-1232

    • ddlscan (in [Tungsten Replicator 6.1 Manual]) now sets the datatype for sequence number columns to BIGINT when generating staging table DDL for Redshift deployments.

      Issues: CT-1235

    • Fixes a situation where tpm update (in [Tungsten Replicator 6.1 Manual]) exits with a Data::Dumper error.

      Issues: CT-1249

  • Core Replicator

    • In heterogeneous replicator deployments, the convertstringfrommysql filter would fail to convert strings for alphanumeric key values.

      Issues: CT-1128

    • Tungsten Replicator could wrongly think it is already in DEGRADED mode when trying to put it into DEGRADED state.

      Issues: CT-1131

    • Tungsten Replicator now recognises Amazon AWS SSL Certificates to enable SSL communication with AWS Aurora.

      Issues: CT-1173

    • When host server time (and thus MySQL time) is not configured as UTC, issuing cluster heartbeat or trepctl heartbeat (in [Tungsten Replicator 6.1 Manual]) in the first hours around daylight savings time would create an invalid time in MySQL.

      For more information on timezones when issuing heartbeats, see trepctl heartbeat Time Zone Handling (in [Tungsten Replicator 6.1 Manual])

      Issues: CT-1174

    • The replicator would fail to apply into Vertica when configured as an offboard installation due to the applier incorrectly expecting the csv files to exist locally on the remote Vertica host.

      Issues: CT-1194

    • Due to a change in the Binary log structure introduced in MySQL 8.0.16, the replicator would fail to extract transactions for Partitioned tables.

      Issues: CT-1201

    • The replicator would fail to correctly parse statements with leading comment blocks in excess of 200 characters.

      Issues: CT-1236