1.41. Tungsten Clustering 5.3.0 GA (12 December 2017)

Version End of Life. 31 July 2020

Release 5.3.0 is a new feature release that contains improvements to the core replicator and manager, including adding new functionality in preparation for the next major release (6.0.0) and future functionality.

Key improvements include:

  • Improved and simplified user-focused logging, to make it easier to identify issues and problems.

  • Easier access to the overall cluster status from the command-line through the Connector cluster-status command.

  • Many fixes and stabilisation improvements to the Connector.

Improvements, new features and functionality

  • Tungsten Connector

    • The connector has been extended to provide cluster status information, and to also to provide this information encapsulated in a JSON format. To get the cluster status through the connector command:

      shell> connector cluster-status

      To get the information in JSON format:

      shell> connector cluster-status -json

      Issues: CONT-630

      For more information, see Connector connector cluster status on the Command-line.

Bug Fixes

  • Behavior Changes

    • The way that information is logged has been improved so that it should be easier to identify and find errors and the causes of errors when looking at the logs. To achieve this, logging is now provided into an additional file, one for each component, and the new files contain only errors at the WARNING or ERROR levels. These files are:

      • manager-user.log

      • connector-user.log

      • replicator-user.log

      These files should be much smaller, and much simpler to read and digest in the event of a problem. Currently the information and warnings added to the logs are being adjusted so that the new log files do not contain unnecessary entries.

      The original log files (tmsvc.log, connector.log, trepsvc.log) remain unchanged in terms of the information logged to them.

      All log files have been updated to ensure that where relevant the service name for the corresponding entry is included. This should further help to identify and pinpoint issues by making it clearer what service triggered a particular logging event.

      Issues: CT-30, CT-69

  • Command-line Tools

    • Backups using datasource backup could fail to complete properly when using xtrabackup.

      Issues: CT-352

    • The tpm diag would fail to get manager logs from hosts that were configured without a replicator, for example standalone connector or witness hosts.

      Issues: CT-360

  • Tungsten Connector

    • If the MySQL server returns a 'too many open connections' error when connecting through the Drizzle driver, the connector could hang with a log message about BufferUnderFlow.

      Issues: CT-86

    • Support for complex passwords within user.map that may include one or more single or double quotes have been updated. The following rules now apply for passwords in user.map:

      • Quotes ' and double quotes " are now supported in the user.map password.

      • If there's a space in the password, the password needs to be surrounded with " or ':

        "password with space"
      • If there's one or several " or ' in the password without space, the password doesn't need to be surrounded

      • If the password itself starts and ends with the same quote (" or '), it needs to be surrounded by quotes.

        "'mypassword'" so that the actual password is 'mypassword'.

      As a general rule, if the password is enclosed in either single or double quotes, these are not included as part of the password during authentication.

      Issues: CONT-239

    • When starting up, the Connector would connect to the first Primary in the first data service within it's own internal list, now the 1st entry of the user.map configuration.

      Issues: CT-385

    • When a connection gets its channel updated by a read/write split (either automatically because Smartscale has been enabled, or manually with selective read/write splitting), the channel that is left in background will be wrongly set as "in use", so the keepalive task won't be able to ping it anymore.

      Issues: CT-388

    • The bridgeServerToClientForcedCloseTimeout property default value has been reduced from 500ms to 50ms.

      Issues: CT-392

      For more information, see Adjusting the Bridge Mode Forced Client Disconnect Timeout.

    • Under certain circumstances it would be possible for the Connector, when configured to choose a Replica based on the Replica latency (i.e. using the --connector-max-slave-latency configuration option), to select the wrong Replica. Rather than choosing the most advanced Replica in terms of the latency, the Replica with the highest latency could be selected instead.

      Issues: CONT-421

    • The connector would log a message each time a connection disappeared without being properly closed. For connections through load balancers this is standard behavior, and could lead to a large number of log entries that would make it difficult to find other errors. The default setting has been changed so the connection warnings are no longer produced by default. This can be changed by setting the printConnectionWarnings property to true.

      Issues: CT-456

  • Tungsten Manager

    • If the manager is on the same host as the coordinator, and there was an error writing information to the disk, and a failover situation occurred, the failiver would not take place. Since a disk write failure is a possible scenario for the the failure to occur, it could lead to the cluster being in an unstable state.

      Issues: CT-364

    • Within a composite deployment, switching a node in a local cluster would cause all relays within the entire composite cluster to point to that node as a Primary datasource.

      Issues: CT-378

Tungsten Clustering 5.3.0 Includes the following changes made in Tungsten Replicator 5.3.0

Release 5.3.0 is an important feature release that contains some key new functionality for replication. In particular:

  • JSON data type column extraction support for MySQL 5.7 and higher.

  • Generated column extraction support for MySQL 5.7 and higher.

  • DDL translation support for heterogeneous targets, initially support DDL translation for MySQL to MySQL, Vertica and Redshift targets.

  • Support for data concentration support for replication into a single target schema (with additional source schema information added to each table) for both HPE Vertica and Amazon Redshift targets.

  • Rebranded and updated support for Oracle extraction with the Oracle Redo Reader, including improvements to offboard deployment, more configuration options, and support for the deployment and installation of multiple offboard replication services within a single replicator.

This release also contains a number of important bug fixes and minor improvements to the product.

Improvements, new features and functionality

  • Behavior Changes

    • The way that information is logged has been improved so that it should be easier to identify and find errors and the causes of errors when looking at the logs. To achieve this, logging is now provided into an additional file, one for each component, and the new files contain only errors at the WARNING or ERROR levels. The new file is replicator-user.log. The original file, trepsvc.log remains unchanged.

      All log files have been updated to ensure that where relevant the service name for the corresponding entry is included. This should further help to identify and pinpoint issues by making it clearer what service triggered a particular logging event.

      Issues: CT-30, CT-69

    • Support for Java 7 (JDK or JRE 1.7) has been deprecated, and will be removed in the 6.0.0 release. The software is compiled using Java 8 with Java 7 compatibility.

      Issues: CT-252

    • Some Javascript filters had DOS style line breaks.

      Issues: CT-376

    • Support for JSON datatypes and generated columns within MySQL 5.7 and greater has been added to the MySQL extraction component of the replicator.


      Due to a MySQL bug, the way that JSON and generated columns is represented within MySQL binary log, it is possible for the size of the data, and the reported size re different and this could cause data corruption To account for this behavior and to prevent data inconsistencies, the replicator can be configured to either ignore, warn, or stop, if the mismatch occurs.

      This can be set by modifying the property replicator.extractor.dbms.json_length_mismatch_policy.

      Until this problem is addressed within MySQL, tpm will still generate a warning about the issue which can be ignored during installation by using the --skip-validation-check=MySQLGeneratedColumnCheck.

      For more information on the effects of the bug, see MySQL Bug #88791.

      Issues: CT-5, CT-468

  • Installation and Deployment

    • The tpm command has been updated to correctly operate with CentOS 7 and higher. Due to an underlying change in the way IP configuration information was sourced, the extraction of the IP address information has been updated to use the ip addr command.

      Issues: CT-35

    • The THL retention setting is now checked in more detail during installation. When the --thl-log-retention is configured when extracting from MySQL, the value is compared to the binary log expiry setting in MySQL (expire_logs_days). If the value is less, then a warning is produced to highlight the potential for loss of data.

      Issues: CT-91

    • A new option, --oracle-redo-temp-tablespace has been added to configure the temporary tablespace within Oracle redo reader extractor deployments.

      Issues: CT-321

  • Command-line Tools

    • 1743

      The sizes outputs for the thl list command, such as -sizes or -sizesdetail command now additionally output summary information for the selected THL events:

      Total ROW chunks: 8 with 7 updated rows (50%)
      Total STATEMENT chunks: 8 with 2552 bytes (50%)
      16 events processed

      A new option has also been added, -sizessummary, that only outputs the summary information.

      Issues: CT-433

      For more information, see thl list -sizessummary Command.

  • Filters

    • A new filter, rowadddbname, has been added to the replicator. This filter adds the incoming schema name, and optional numeric hash value of the schema, to every row of THL row-based changes. The filter is designed to be used with heterogeneous and analytics applications where data is being concentrated into a single schema and where the source schema name will be lost during the concentration and replication process.

      In particular, it is designed to work in harmony with the new Redshift and Vertica based single-schema appliers where data from multiple, identical, schemas are written into a single target schema for analysis.

      Issues: CT-98

    • A new filter has been added, rowadddbname, which adds the source database name and optional database hash to every incoming row of data. This can be used to help identify source information when concentrating information into a single schema.

      Issues: CT-407

Bug Fixes

  • Installation and Deployment

    • An issue has been identified with the way certain operating systems now configure their open files limits, which can upset the checks within tpm that determine the open files limits configured for MySQL. To ensure that the open files limit has been set correctly, check the configuration of the service:

      1. Copy the system configuration:

        shell> sudo cp /lib/systemd/system/mysql.service /etc/systemd/system/
        shell> sudo vim /etc/systemd/system/mysql.service
      2. Add the following line to the end of the copied file:

      3. Reload the systemctl daemon:

        shell> sudo systemctl daemon-reload
      4. Restart MySQL:

        shell> service mysql restart

      That configures everything properly and MySQL should now take note of the open_files_limit config option.

      Issues: CT-148

    • The check to determine if triggers had been enabled within the MySQL data source would not get executed correctly, meaning that warnings about unsupported triggers would not trigger a notification.

      Issues: CT-185

    • When using tpm diag on a MySQL deployment, the MySQL error log would not be identified and included properly if the default datadir option was not /var/lib/mysql.

      Issues: CT-359

    • Installation when enabling security through SSL could fail intermittently during installation because the certificates would fail to get copied to the required directory during the installation process.

      Issues: CT-402

    • The Net::SSH libraries used by tpm have been updated to reflect the deprecation of paranoid parameter.

      Issues: CT-426

    • Using a complex password, particularly one with single or double quotes, when specifying a password for tpm, could cause checks and the installation to raise errors or fail, although the actual configuration would work properly. The problem was limited to internal checks by tpm only.

      Issues: CT-440

  • Command-line Tools

    • The startall command would fail to correctly start the Oracle redo reader process.

      Issues: CT-283

    • The tpm command would fail to remove the Oracle redo reader user when using tpm uninstall.

      Issues: CT-299

    • The replicator stop command would not stop the Oracle redo reader process.

      Issues: CT-300

    • Within Vertica deployments, the internal identity of the applier was set incorrectly to PostgreSQL. This would make it difficult for certain internal processes to identify the true datasource type. The setting did not affect the actual operation.

      Issues: CT-452

  • Core Replicator

    • When parsing THL data it was possible for the internal THL processing to lead to a java.util.ConcurrentModificationException. This indicated that the underlying THL event metadata structure used internally had changed between uses.

      Issues: CT-355