1.6. Tungsten Clustering 6.1.2 GA (20 January 2020)

Version End of Life. Not Yet Set

Release 6.1.2 contains significant improvements as well as some needed bugfixes.

Behavior Changes

The following changes have been made to Tungsten Cluster and may affect existing scripts and integration tools. Any scripts or environment which make use of these tools should check and update for the new configuration:

  • Behavior Changes

    • The Passive Witness functionality is now officially DEPRECATED and will be REMOVED in version 6.2

      Issues: CT-653

  • Installation and Deployment

    • In cluster deployments with witness nodes, if the MySQL servers had been configured to listen on any port other than the standard 3306, the tpm (in [Tungsten Clustering (for MySQL) 6.1 Manual]) command would default to the wrong port number.

      When witness hosts are in use, the tpm (in [Tungsten Clustering (for MySQL) 6.1 Manual]) command will no longer "guess" the data source port automatically by running my_print_defaults.

      It is now mandatory to specify the MySQL data server port explicitly using --datasource-port={mysql_listen_port} (in [Tungsten Clustering (for MySQL) 6.1 Manual]) or one of its aliases

      Issues: CT-1071

Improvements, new features and functionality

  • Command-line Tools

    • The tpm (in [Tungsten Clustering (for MySQL) 6.1 Manual]) command was originally written in Ruby. This improvement converts tpm to Perl over time, starting with the tpm shell wrapper and refactoring each sub-command one-by-one.

      For this release, we have redone the diag, mysql and connector sub-commands.

      This also wraps the update sub-command to provide the CT-1093 clustering fix.

      Issues: CT-1048

    • This version includes an update to the new BETA tool to provision Replicas, and Primaries in a Composite Active/Active topology. This release fully supports provisioning of nodes in a Composite Active/Active Topology.

      It can be invoked by running prov-sl.sh or tps.pl.

      This tool will replace tungsten_provision_slave (in [Tungsten Clustering (for MySQL) 6.1 Manual]) in a future release.

      Issues: CT-1070

    • Added the tpm policy (in [Tungsten Clustering (for MySQL) 6.1 Manual]) subcommand to allow easy get and set cluster policy operations instead of using cctrl.

      For more information, please see ...

      Issues: CT-1106

    • The new changes made to the tpm (in [Tungsten Clustering (for MySQL) 6.1 Manual]) command, require that the zip package be installed on all DB hosts

      Issues: CT-1111

  • Tungsten Manager

    • The new default datasources sort order is alphabetically when using the cctrl ls command. Additionally, the sort order of the datasources list is now configurable.

      The behavior is controlled by the tpm configuration property cctrl.sort.datasources.alphabetically which has a default value of true (meaning alpha sort).

      If set to false, the sort is ordered by datasource role, so the Primary or relay will appear first, followed by the Replicas. For example, use the following in the INI file for role-based sorting:

      property=cctrl.sort.datasources.alphabetically=false

      Issues: CT-1018

Bug Fixes

  • Installation and Deployment

    • Ensure that all Connector-Manager communications are SSL-encrypted when --disable-security-controls=false (in [Tungsten Clustering (for MySQL) 6.1 Manual])

      Issues: CT-1060

  • Command-line Tools

    • When performing an update of a cluster with tpm (in [Tungsten Clustering (for MySQL) 6.1 Manual]), the cluster would be switched to MAINTENANCE (in [Tungsten Clustering (for MySQL) 6.1 Manual]) but would remain in this policy after the update. The original policy is now retained during the update.

      Issues: CT-595, CT-1093

    • The deployall (in [Tungsten Clustering (for MySQL) 6.1 Manual]) script was only able to install init.d system startup scripts.

      In this release, the script will now detect the initialization system in use (systemd or initd) and prefer systemd when both are available.

      For systemd configurations only:

      For continuity of service reasons, the deployall (in [Tungsten Clustering (for MySQL) 6.1 Manual]) script does NOT restart individual components when called, it will only install systemd scripts. This implies that, right after a call to deployall and before host restart, the system will stay in a mixed mode where systemd scripts are in place but components were started without systemd, so won’t be controllable by it.

      In order to align the configuration, you will need to run

      shell> component stop sysd
      shell> sudo systemctl start tcomponent

      For example:

      shell> connector stop sysd
      shell> sudo systemctl start tconnector

      Issues: CT-853

    • When issuing tpm connector --samples (in [Tungsten Clustering (for MySQL) 6.1 Manual]), the output displayed clear text passwords. In this release the passwords are obsfucated.

      Issues: CT-1021

    • Continuent Tungsten Clustering now uses the xtrabackup command instead of the deprecated innobackupex to create and restore backups. A new check was added to TPM for validating different xtrabackup versions along with MySQL version compatibility. The oldest supported version of xtrabackup is v2.3

      Issues: CT-1074

  • Core Replicator

    • When configuring SSL for the Connector only, the Replicator would fail to start due to the Replicator also looking for the SSL configuration.

      Issues: CT-956

  • Tungsten Connector

    • Fixed an issue where some applications might fail to connect to the Connector with MariaDB 10+

      Previously, when using MariaDB 10+, the Connector would be confused by the 10 and will think it is a MySQL 8+ server. By default, the Connector will offer to connect with caching_sha2_password. If the application does not know how to switch authentication plugins, it would fail with a message similar to the following:

      The server requested authentication method unknown to the client [caching_sha2_password]

      The previous work-around was to specify the authentication plugin using the tpm (in [Tungsten Clustering (for MySQL) 6.1 Manual]) command:

      --property=defaultAuthPlugin=mysql_native_password

      Issues: CT-1033

    • Improved Tungsten Connector bridge mode performance when transferring small amounts of data.

      Issues: CT-1081

  • Tungsten Manager

    • When using the cctrl (in [Tungsten Clustering (for MySQL) 6.1 Manual]) command interactively, the `cluster topology` TAB completion was showing invalid options. Invalid options have been removed.

      Issues: CT-979

    • Fixed an issue where long-duration operations like failover and switch would create false positives about network partitioning after completion.

      Issues: CT-1023

    • Fixed the cctrl (in [Tungsten Clustering (for MySQL) 6.1 Manual]) command so that the '[SSL]' indicator in the `ls` output is displayed. This is a Version 5 feature that was lost in v6.0.0, now restored.

      Issues: CT-1061

    • Fixed the cctrl (in [Tungsten Clustering (for MySQL) 6.1 Manual]) command `datasource {hostname} restore` which was failing in Composite Active/Active cluster deployments with: ERROR: MORE THAN ONE PRIMARY DATA SOURCE FOUND.

      Issues: CT-1062

    • Continuent Tungsten Clustering now only checks for a running MySQL server when the backup method is 'mysqldump' in cctrl (in [Tungsten Clustering (for MySQL) 6.1 Manual]).

      Background: Running datasource {hostname} restore inside cctrl (in [Tungsten Clustering (for MySQL) 6.1 Manual]) would fail when the MySQL server was not running. Only the 'mysqldump' method requires a running MySQL server. The 'xtrabackup-full' and 'xtrabackup-incremental' methods will work even if MySQL is stopped.

      Issues: CT-1077

    • tungsten_find_orphaned (in [Tungsten Clustering (for MySQL) 6.1 Manual]) was displaying an incorrect error message if a service name wasn't supplied correctly.

      Issues: CT-1079

    • tungsten_find_orphaned (in [Tungsten Clustering (for MySQL) 6.1 Manual]) would error with 'Argument "" isn't numeric in addition'.

      Issues: CT-1080

    • Fixed an issue where Composite clusters with only a single site would come up as SHUNNED after install.

      Issues: CT-1101

Tungsten Clustering 6.1.2 Includes the following changes made in Tungsten Replicator 6.1.2

Release 6.1.2 contains both significant improvements as well as some needed bugfixes.

Behavior Changes

The following changes have been made to Tungsten Replicator and may affect existing scripts and integration tools. Any scripts or environment which make use of these tools should check and update for the new configuration:

  • Behavior Changes

    • Certified the Tungsten product suite with Java 11.

      A small set of minor issues have been found and fixed (CT-1091, CT-1076) along with this certification.

      The code is now compiled with Java compiler v11 while keeping Java 8 compatibility.

      Java 9 and 10 have been tested and validated but certification and support will only cover Long Term releases.

      Note

      Known Issue

      With Java 11, command line tools are slower. There is no impact on the overall clustering or replication performance but this can affect manual operations using CLI tools such as cctrl and trepctl (in [Tungsten Replicator 6.1 Manual])

      Issues: CT-1052

Improvements, new features and functionality

  • Core Replicator

    • A new Replicator role, thl-server, has been added.

      This new feature allows your Replica replicators to still pull generated THL from a Primary even when the Primary replicator has stopped extracting from the binlogs.

      If used in Tungsten Clustering, this feature must only be enabled when the cluster is in MAINTENANCE mode.

      Issues: CT-58

      For more information, see Understanding Replicator Roles (in [Tungsten Replicator 6.1 Manual]).

    • A new JavaScript filter dropddl.js (in [Tungsten Replicator 6.1 Manual]) has been added to allow selective removal of specific object DDL from THL.

      Issues: CT-1092

Bug Fixes

  • Behavior Changes

    • If you need to reposition the extractor, there are a number of ways to do this, including the use of the options -from-event or -base-seqno

      Both of these options are mutually exclusive, however in some situations, such as when positioning against an Aurora source, you may need to issue both of these options together. Previously this was not possible. In this release both options can now be supplied providing that you include the additional -force option, for example

      shell> trepctl -service serviceName online -base-seqno 53 -from-event 000412:762897 -force

      Issues: CT-1065

    • When the Replicator inserts a heartbeart there is an associated timezone. Previously, the heartbeat would be inserted using the GMT timezone, which fails during the DST switch window. The new default uses the Replicator host's timezone instead.

      This defaults change corrects an edge case where inserting a heartbeat will fail during the DST switch window when the MYSQL server is running in a different timezone than the Replicator (which runs in GMT).

      For example, on 31th March 2019, the time switch occurred @ 2AM in the Europe/Paris timezone. When inserting a heartbeat in the window between 4 and 5 AM (say at 4:15am), the corresponding GMT time would be 2:15am, which is invalid in the Europe/Paris timezone. Replicator would then fail if the MySQL timezone was set to Europe/Paris, as it would try to insert an invalid timestamp.

      A new option, -tz has been added into the trepctl heartbeat (in [Tungsten Replicator 6.1 Manual]) command to force the use of a specific timezone.

      For example, use GMT as the timezone when inserting a heartbeat:

      shell> trepctl heartbeat -tz NONE

      Use the Replicator host's timezone to insert the heartbeat:

      shell> trepctl heartbeat -tz HOST

      Use the given timezone to insert the heartbeat:

      shell> trepctl heartbeat -tz {valid timezone id}

      If the MySQL server timezone is different from the host timezone (which is strongly not recommended), then -tz {valid timezone id} should be used instead where {valid timezone id} should be the same as the MySQL server timezone.

      Issues: CT-1066

    • Corrected resource leak when loading Java keystores

      Issues: CT-1091

  • Command-line Tools

    • Fixed error message to indicate the need to specify a service on Composite Active/Active clusters for the tungsten_find_position and tungsten_find_seqno commands.

      Issues: CT-1098

    • The tpm (in [Tungsten Replicator 6.1 Manual]) command no longer reports warnings about existing system triggers with MySQL 8+

      Issues: CT-1099

  • Core Replicator

    • When configuring a Kafka Applier, the Kafka Port was set incorrectly

      Issues: CT-693

    • If a JSON field contained a single quote, the replicator would break during the apply stage whilst running the generated SQL into MySQL.

      Single quotes will now be properly escaped to solve this issue

      Issues: CT-983

    • Under rare circumstances (network packet loss or MySQL Server hang), the replicator would also hang until restarted.

      This issue has been fixed by using specific network timeouts in both the replicator and in the Drizzle jdbc driver connection logic

      Issues: CT-1034

    • When configuring Active/Active, standalone replicators, with the BidiSlave filter enabled, the replicator was incorrectly parsing certain DDL Statements and marking them as unsafe, as a result they were being dropped by the applier and ignored

      The full list of DDL commands fixed in this release are as follows:

      • CREATE|DROP TRIGGER

      • CREATE|DROP FUNCTION

      • CREATE|DROP|ALTER|RENAME USER

      • GRANT|REVOKE

      Issues: CT-1084, CT-1117

    • The following warnings would appear in the replicator log due to GTID events not being handled.

      WARN extractor.mysql.LogEvent Skipping unrecognized binlog event type 33, 34 or 35)

      The WARN message will no longer appear, however GTID Events are still not handled in this release, but will be fully extracted in a future release.

      Issues: CT-1114