2.8. Tungsten Replicator 5.4.0 GA (31 July 2019)

Version End of Life. Not Yet Set

Release 5.4.0 contains both significant improvements as well as some needed bugfixes. One of the main features of this release is MySQL 8 support.

Improvements, new features and functionality

  • Command-line Tools

    • Two new utility scripts have been added to the release to help with setting the Replicator position:

      - tungsten_find_position, which assists with locating information in the THL based on the provided MySQL binary log event position and outputs a dsctl set (in [Tungsten Replicator 5.4 Manual]) command as output.

      - tungsten_find_seqno, which assists with locating information in the THL based on the provided sequence number and outputs a dsctl set (in [Tungsten Replicator 5.4 Manual]) command as output.

      Issues: CT-934

  • Core Replicator

    • The replicator has been updated to support the new character sets supported by MySQL 5.7 and MySQL 8.0, including the UTF-8-mb4 series.

      Issues: CT-700, CT-970

    • A new, beta-quality command has been included called prov-sl.sh which is intended to eventually replace the current tungsten_provision_slave (in [Tungsten Replicator 5.4 Manual]) script.

      Currently, prov-sl.sh supports provisioning slaves using mysqldump and xtrabackup tools, and is MySQL 8-compatible. 

      The prov-sl.sh command is written in Bash, has less dependencies compared to the current version and is meant to fix a number of issues with the current version.

      Backups are streamed from source to target so that an intermediate write to disk is not performed, resulting in faster provisioning times.

      Logs are written to $CONTINUENT_ROOT/service_logs/prov-sl.log (i.e. /opt/continuent/service_logs/prov-sl.log).

      For example, provision a slave from [source db] using mysqldump (default):

      shell> prov-sl.sh -s {source db}

      As another example, use xtrabackup for the backup method, with 10 parallel threads (default is 4), and ssh is listening on port 2222:

      shell> prov-sl.sh -s {source db} -n xtrabackup -t 10 -p 2222

      Warning

      At the moment, prov-sl.sh does not support MultiMaster topologies when used with Tungsten Clustering, however it will be included in a future release.

      Issues: CT-614, CT-723, CT-809, CT-855, CT-963

    • Upgraded the Drizzle driver to support MySQL 8 authentication protocols (SHA256, caching_sha2).

      Issues: CT-914, CT-931, CT-966

    • The Redshift Applier now allows AWS authentication using IAM Roles. Previously authentication was possible via Access and Secret Key pairs only.

      Issues: CT-980

      For more information, see Redshift Preparation for Amazon Redshift Deployments (in [Tungsten Replicator 5.4 Manual]).

Bug Fixes

  • Command-line Tools

    • The --hosts (in [Tungsten Replicator 5.4 Manual]) option was not working with the diag sub-command of the tpm (in [Tungsten Replicator 5.4 Manual]) command on nodes installed using the INI method.

      The corrected behavior is as follows:

      • With Staging-method deployments, the tpm diag (in [Tungsten Replicator 5.4 Manual]) command continues to behave as before:

        • The tpm diag (in [Tungsten Replicator 5.4 Manual]) command alone will obtain diagnostics from all hosts in the cluster.

        • The tpm diag --hosts host1,host2,hostN command will obtain diagnostics from the specified host(s) only.

      • With INI-method deployments, the new behavior is as follows:

        • The tpm diag (in [Tungsten Replicator 5.4 Manual]) command alone will obtain diagnostics from the local host only.

        • The tpm diag --hosts host1,host2,hostN command will obtain diagnostics from the specified host(s) only.

          Warning

          Limitation: the host list MUST include the local hostname or the command will fail.

      Issues: CT-345

    • When using tpm (in [Tungsten Replicator 5.4 Manual]) with the INI method, the command would search multiple locations for suitable INI files. This could lead to multiple definitions of the same service, which could in turn lead to duplication of the installation process and occasional failures. If multiple INI files are found, a warning is now produced to highlight the potential for failures.

      Issues: CT-626

    • The trepctl (in [Tungsten Replicator 5.4 Manual]) command now properly handles the -all-services option for the reset sub-command.

      Issues: CT-762

    • The command tpm reverse --ini-format now outputs correctly (without the double-dashes and the trailing backslash).

      Issues: CT-827, CT-877

    • The tpm (in [Tungsten Replicator 5.4 Manual]) command now properly handles network interface names containing colons and/or dots.

      Issues: CT-864

    • When executing mysqldump, all Tungsten tools no longer use the --add-drop-database flag as it will prevent MySQL 8+ from restoring the dump.

      Issues: CT-935

  • Core Replicator

    • Added support for missing charset GB18030 to correct WARN extractor.mysql.MysqlBinlog Unknown charset errors.

      Issues: CT-915, CT-932

    • Loading data into Redshift would fail with the following error if a row of data contained a specific control character (0x00 (NULL))

      Missing newline: Unexpected character 0x30 found at location nnn

      Issues: CT-984

    • Now properly extracting the Geometry datatype.

      Issues: CT-997

    • The ddl_map.json file used by the apply_schema_changes filter was missing a rule to handle ALTER TABLE statements when replicating between MySQL and Redshift

      Issues: CT-1002

    • The extract_schema_change filter wasn't escaping " (double-quotes) and the generated JSON would then cause the applier to error with

      pendingExceptionMessage: SyntaxError: missing } after property list »
      (../../tungsten-replicator//support/filters-javascript/apply_schema_changes.js#236(eval)#1)

      Issues: CT-1011