B.2. Tungsten Replicator 4.0.6 GA (8 Dec 2016)

Tungsten Replicator 4.0.6 is a bugfix release that contains critical fixes and improvements to the Tungsten Replicator 4.0.5 release.

Behavior Changes

The following changes have been made to Tungsten Replicator and may affect existing scripts and integration tools. Any scripts or environment which make use of these tools should check and update for the new configuration:

  • For compatibility with MySQL 5.7, the tpm command will now check for the super_read_only setting and warn if this setting is enabled.

    Issues: CONT-1039

  • For compatibility with MySQL 5.7, the tpm command will use the authentication_string field for validating passwords.

    Issues: CONT-1058

  • For compatibility with MySQL 5.7, the tpm command will now ignore the sys schema.

    Issues: CONT-1059

Known Issues

The following issues may affect the operation of Tungsten Replicator and should be taken into account when deploying or updating to this release.

  • Installation and Deployment

    • When running tpm update preperties set during the initial install could be reset or changed to their default value.

      Issues: CONT-1579

  • Command-line Tools

    • Running multi_trepctl in a multi-site, multi-master (MSMM) deployment could fail to report all of the running replication processes.

      Issues: CONT-1585

  • Core Replicator

    • There is a limit in the communication protocal for the replicator which limits the number of fragments within a single transaction in the THL to 32768. Although this is not a limit in the THL format, it is a limit in the protocol used to exchanged the THL information between replicators.

      The size of this value, and therefore, the maximum number of fragments cannot be increased without creating an incompatible change within the replicator. This creates a limit to the maximum size of a single transaction that can be replicated. Although this figure cannot be altered, the size of each individual fragment can be increased. The default setting is 1,000,000, creating a limit of approximately 32GB.

      To increase the fragment size, set the value of the property replicator.extractor.dbms.transaction_frag_size. For example, increasing the value to 2,000,000 would increase the maximum THL transaction size to approximately 64GB.

      Care should be taken when increasing this value, as it also increases the amount of memory required to handle the transaction.

      Issues: CONT-1574

  • Filters

    • There is a known issue with the fixmysqlstrings.js filter. When translating BINARY or VARBINARY datatypes into a hex value, if the encoding set for the MySQL and replicator instance is not UTF-8, an implied character set conversion can take place. This leads to a corruption of the information when it is turned into a hex string. This is due to limitations of the internal datatypes available within the JavaScript environment used for the translation.

      Issues: CONT-1508

Improvements, new features and functionality

  • Installation and Deployment

    • Due to changes in the datatypes available in MySQL 5.7 and the supported datatypes within Tungsten Replicator, and coinciding with changes to the way this information is available, the tpm checks for compatibility may no longer highlight important option changes. For example, virtual columns and JSON columns in MySQL 5.7 are not replicated. During installation, if tpm identifies that MySQL 5.7 is in use, the following message will be reported:

      IMPORTANT: The replicator is unable to replicate tables that have
      columns defined as type JSON or that utilise VIRTUAL GENERATED values!
      The use of these features will cause replication to fail. If you want
      tpm to check for these add --mysql-allow-intensive-checks to the
      configuration. Be aware that the checks will query the
      information_schema and if you have thousands of tables this may affect
      other queries while the check runs. Otherwise, if you have confirmed
      manually that JSON or VIRTUAL GENERATED columns are not being used,
      you can skip this check by
      adding --skip-validation-check=MySQLUnsopportedDataTypesCheck to your

      To address this issue, when using tpm during an installation, more intensive checks for tables with unsupported types can be performed. For example, when checking the special column types used in all tables within an existing installation, tpm must check each table individually. As this can increase the load on the server during installagtion, tpm by default does not perform these checks. Instead, these checks can be enabled by using the --mysql-allow-intensive-checks option during configuration. Enabling this option provides for a much more detailed check, but may cause the installation process to take longer.

      Issues: CONT-1551, CONT-1576

  • Core Replicator

    • If the slave THL file ends with an event that was ultimately filtered, and the the replicator master and slave roles are then switched, the new master could generate an incorrect sequence number.

      Issues: CONT-1545

Bug Fixes

  • Installation and Deployment

    • The Ruby Net::SSH libraries used by tpm have been updated to the latest version. This addresses issues with SSH and staging based deployments, including KEX algorithm errors.

      Issues: CT-16

    • The built-in check for InnoDB did not work for MySQL 5.6 and could fail to identify InnoDB support on the MySQL server.

      Issues: CONT-1577

  • Core Replicator

    • Extraction from the MySQL binary log would fail if the binary log event ID is bigger than a Java Int. This could be triggered if a large (greater than 2B) transaction is inserted into the binary log.

      Issues: CONT-1541