A.2. Tungsten Clustering 6.1.2 GA (20 January 2020)

Version End of Life. Not Yet Set

Release 6.1.2 contains significant improvements as well as some needed bugfixes.

Behavior Changes

The following changes have been made to Tungsten Clustering and may affect existing scripts and integration tools. Any scripts or environment which make use of these tools should check and update for the new configuration:

  • The Passive Witness functionality is now officially DEPRECATED and will be REMOVED in version 6.2

    Issues: CT-653

  • In cluster deployments with witness nodes, if the MySQL servers had been configured to listen on any port other than the standard 3306, the tpm command would default to the wrong port number.

    When witness hosts are in use, the tpm command will no longer "guess" the data source port automatically by running my_print_defaults.

    It is now mandatory to specify the MySQL data server port explicitly using --datasource-port={mysql_listen_port} or one of its aliases

    Issues: CT-1071

Improvements, new features and functionality

  • Command-line Tools

    • The tpm command was originally written in Ruby. This improvement converts tpm to Perl over time, starting with the tpm shell wrapper and refactoring each sub-command one-by-one.

      For this release, we have redone the diag, mysql and connector sub-commands.

      This also wraps the update sub-command to provide the CT-1093 clustering fix.

      Issues: CT-1048

    • This version includes an update to the new BETA tool to provision slaves, and masters in a Composite MultiMaster topology. This release fully supports provisioning of nodes in a Composite Master-Master Topology.

      It can be invoked by running prov-sl.sh or tps.pl.

      This tool will replace tungsten_provision_slave in a future release.

      Issues: CT-1070

    • Added the tpm policy subcommand to allow easy get and set cluster policy operations instead of using cctrl.

      For more information, please see ...

      Issues: CT-1106

    • The new changes made to the tpm command, require that the zip package be installed on all DB hosts

      Issues: CT-1111

  • Tungsten Manager

    • The new default datasources sort order is alphabetically when using the cctrl ls command. Additionally, the sort order of the datasources list is now configurable.

      The behavior is controlled by the tpm configuration property cctrl.sort.datasources.alphabetically which has a default value of true (meaning alpha sort).

      If set to false, the sort is ordered by datasource role, so the master or relay will appear first, followed by the slaves. For example, use the following in the INI file for role-based sorting:

      property=cctrl.sort.datasources.alphabetically=false

      Issues: CT-1018

Bug Fixes

  • Installation and Deployment

    • Ensure that all Connector-Manager communications are SSL-encrypted when --disable-security-controls=false

      Issues: CT-1060

  • Command-line Tools

    • When performing an update of a cluster with tpm, the cluster would be switched to MAINTENANCE but would remain in this policy after the update. The original policy is now retained during the update.

      Issues: CT-595, CT-1093

    • The deployall script was only able to install init.d system startup scripts.

      In this release, the script will now detect the initialization system in use (systemd or initd) and prefer systemd when both are available.

      For systemd configurations only:

      For continuity of service reasons, the deployall script does NOT restart individual components when called, it will only install systemd scripts. This implies that, right after a call to deployall and before host restart, the system will stay in a mixed mode where systemd scripts are in place but components were started without systemd, so won’t be controllable by it.

      In order to align the configuration, you will need to run

      shell> component stop sysd
      shell> sudo systemctl start tcomponent

      For example:

      shell> connector stop sysd
      shell> sudo systemctl start tconnector

      Issues: CT-853

    • When issuing tpm connector --samples, the output displayed clear text passwords. In this release the passwords are obsfucated.

      Issues: CT-1021

    • Continuent Tungsten Clustering now uses the xtrabackup command instead of the deprecated innobackupex to create and restore backups. A new check was added to TPM for validating different xtrabackup versions along with MySQL version compatibility. The oldest supported version of xtrabackup is v2.3

      Issues: CT-1074

  • Core Replicator

    • When configuring SSL for the Connector only, the Replicator would fail to start due to the Replicator also looking for the SSL configuration.

      Issues: CT-956

  • Tungsten Connector

    • Fixed an issue where some applications might fail to connect to the Connector with MariaDB 10+

      Previously, when using MariaDB 10+, the Connector would be confused by the 10 and will think it is a MySQL 8+ server. By default, the Connector will offer to connect with caching_sha2_password. If the application does not know how to switch authentication plugins, it would fail with a message similar to the following:

      The server requested authentication method unknown to the client [caching_sha2_password]

      The previous work-around was to specify the authentication plugin using the tpm command:

      --property=defaultAuthPlugin=mysql_native_password

      Issues: CT-1033

    • Improved Tungsten Connector bridge mode performance when transferring small amounts of data.

      Issues: CT-1081

  • Tungsten Manager

    • When using the cctrl command interactively, the `cluster topology` TAB completion was showing invalid options. Invalid options have been removed.

      Issues: CT-979

    • Fixed an issue where long-duration operations like failover and switch would create false positives about network partitioning after completion.

      Issues: CT-1023

    • Fixed the cctrl command so that the '[SSL]' indicator in the `ls` output is displayed. This is a Version 5 feature that was lost in v6.0.0, now restored.

      Issues: CT-1061

    • Fixed the cctrl command `datasource {hostname} restore` which was failing in Composite Master/Master cluster deployments with: ERROR: MORE THAN ONE PRIMARY DATA SOURCE FOUND.

      Issues: CT-1062

    • Continuent Tungsten Clustering now only checks for a running MySQL server when the backup method is 'mysqldump' in cctrl.

      Background: Running datasource {hostname} restore inside cctrl would fail when the MySQL server was not running. Only the 'mysqldump' method requires a running MySQL server. The 'xtrabackup-full' and 'xtrabackup-incremental' methods will work even if MySQL is stopped.

      Issues: CT-1077

    • tungsten_find_orphaned was displaying an incorrect error message if a service name wasn't supplied correctly.

      Issues: CT-1079

    • tungsten_find_orphaned would error with 'Argument "" isn't numeric in addition'.

      Issues: CT-1080

    • Fixed an issue where Composite clusters with only a single site would come up as SHUNNED after install.

      Issues: CT-1101

Tungsten Clustering 6.1.2 Includes the following changes made in Tungsten Replicator 6.1.2

Release 6.1.2 contains both significant improvements as well as some needed bugfixes.

Behavior Changes

The following changes have been made to Continuent Replicator and may affect existing scripts and integration tools. Any scripts or environment which make use of these tools should check and update for the new configuration:

  • Certified the Tungsten product suite with Java 11.

    A small set of minor issues have been found and fixed (CT-1091, CT-1076) along with this certification.

    The code is now compiled with Java compiler v11 while keeping Java 8 compatibility.

    Java 9 and 10 have been tested and validated but certification and support will only cover Long Term releases.

    Note

    Known Issue

    With Java 11, command line tools are slower. There is no impact on the overall clustering or replication performance but this can affect manual operations using CLI tools such as cctrl and trepctl

    Issues: CT-1052

Improvements, new features and functionality

  • Core Replicator

    • A new Replicator role, thl-server, has been added.

      This new feature allows your slave replicators to still pull generated THL from a Master even when the Master replicator has stopped extracting from the binlogs.

      If used in Tungsten Clusters, this feature must only be enabled when the cluster is in MAINTENANCE mode. For more details refer to ???

      Issues: CT-58

    • A new JavaScript filter dropddl.js has been added to allow selective removal of specific object DDL from THL.

      Issues: CT-1092

Bug Fixes

  • Behavior Changes

    • If you need to reposition the extractor, there are a number of ways to do this, including the use of the options -from-event or -base-seqno

      Both of these options are mutually exclusive, however in some situations, such as when positioning against an Aurora source, you may need to issue both of these options together. Previously this was not possible. In this release both options can now be supplied providing that you include the additional -force option, for example

      shell> trepctl -service serviceName online -base-seqno 53 -from-event 000412:762897 -force

      Issues: CT-1065

    • When the Replicator inserts a heartbeart there is an associated timezone. Previously, the heartbeat would be inserted using the GMT timezone, which fails during the DST switch window. The new default uses the Replicator host's timezone instead.

      This defaults change corrects an edge case where inserting a heartbeat will fail during the DST switch window when the MYSQL server is running in a different timezone than the Replicator (which runs in GMT).

      For example, on 31th March 2019, the time switch occurred @ 2AM in the Europe/Paris timezone. When inserting a heartbeat in the window between 4 and 5 AM (say at 4:15am), the corresponding GMT time would be 2:15am, which is invalid in the Europe/Paris timezone. Replicator would then fail if the MySQL timezone was set to Europe/Paris, as it would try to insert an invalid timestamp.

      A new option, -tz has been added into the trepctl heartbeat command to force the use of a specific timezone.

      For example, use GMT as the timezone when inserting a heartbeat:

      shell> trepctl heartbeat -tz NONE

      Use the Replicator host's timezone to insert the heartbeat:

      shell> trepctl heartbeat -tz HOST

      Use the given timezone to insert the heartbeat:

      shell> trepctl heartbeat -tz {valid timezone id}

      If the MySQL server timezone is different from the host timezone (which is strongly not recommended), then -tz {valid timezone id} should be used instead where {valid timezone id} should be the same as the MySQL server timezone.

      Issues: CT-1066

    • Corrected resource leak when loading Java keystores

      Issues: CT-1091

  • Command-line Tools

    • Fixed error message to indicate the need to specify a service on Composite Master/Master clusters for the tungsten_find_position and tungsten_find_seqno commands.

      Issues: CT-1098

    • The tpm command no longer reports warnings about existing system triggers with MySQL 8+

      Issues: CT-1099

  • Core Replicator

    • When configuring a Kafka Applier, the Kafka Port was set incorrectly

      Issues: CT-693

    • If a JSON field contained a single quote, the replicator would break during the apply stage whilst running the generated SQL into MySQL.

      Single quotes will now be properly escaped to solve this issue

      Issues: CT-983

    • Under rare circumstances (network packet loss or MySQL Server hang), the replicator would also hang until restarted.

      This issue has been fixed by using specific network timeouts in both the replicator and in the Drizzle jdbc driver connection logic

      Issues: CT-1034

    • When configuring MultiMaster, standalone replicators, with the BidiSlave filter enabled, the replicator was incorrectly parsing certain DDL Statements and marking them as unsafe, as a result they were being dropped by the applier and ignored

      The full list of DDL commands fixed in this release are as follows:

      • CREATE|DROP TRIGGER

      • CREATE|DROP FUNCTION

      • CREATE|DROP|ALTER|RENAME USER

      • GRANT|REVOKE

      Issues: CT-1084, CT-1117

    • The following warnings would appear in the replicator log due to GTID events not being handled.

      WARN extractor.mysql.LogEvent Skipping unrecognized binlog event type 33, 34 or 35)

      The WARN message will no longer appear, however GTID Events are still not handled in this release, but will be fully extracted in a future release.

      Issues: CT-1114