Continuent Documentation

Search:
Manual Home
Release Notes

1.2. Tungsten Clustering 6.1.0 GA (31 July 2019)

Version End of Life. Not Yet Set

Release 6.1.0 contains both significant improvements as well as some needed bugfixes. One of the main features of this release is MySQL 8 support.

The Tungsten Stack now supports the new MySQL 8.0 authentication plugins. Both sha256_password and caching_sha2_password (the new default) are supported by the Replicator, Manager and Connector.

More info on these authentication plugins can be found here: https://dev.mysql.com/doc/refman/8.0/en/sha256-pluggable-authentication.html

The Drizzle driver has been updated to support these new authentication methods, and the MySQL Connector/J 8 is also supported.

Behavior Changes

The following changes have been made to Continuent Tungsten and may affect existing scripts and integration tools. Any scripts or environment which make use of these tools should check and update for the new configuration:

  • The Connector passThroughMode configuration option is now deprecated.

    The following passThroughMode entry will be removed from tungsten-connector/conf/connector.properties. There is currently no tpm (in [Tungsten Clustering (for MySQL) 6.1 Manual]) option for this, and it is undocumented. The default will be kept to passThroughMode=true.

    # The Tungsten Connector offers an extra fast data transfer mode known as
    # pass-through. When the following switch enabled (default), the Connector
    # will directly transfer data packets between the client and the server.
    # When disabled, every native MySQL command will be translated into a JDBC call.
    passThroughMode=true

    Issues: CT-897

Known Issue

The following issues may affect the operation of Continuent Tungsten and should be taken into account when deploying or updating to this release.

  • Some applications might fail to connect to the Connector with MariaDB 10+

    When using MariaDB 10+, the Connector will be confused by the 10 and will think it is a MySQL 8+ server. By default, the Connector will offer to connect with caching_sha2_password. If the application does not know how to switch authentication plugins, it will fail with a message similar to the following:

    The server requested authentication method unknown to the client [caching_sha2_password]

    As a work-around, you may specify the authentication plugin using the following tpm (in [Tungsten Clustering (for MySQL) 6.1 Manual]) command option:

    --property=defaultAuthPlugin=mysql_native_password

    Issues: CT-1033

Improvements, new features and functionality

  • Command-line Tools

    • A new utility script has been added to the release, tungsten_post_process (in [Tungsten Clustering (for MySQL) 6.1 Manual]), which assists with the graceful maintenance of the static cross-site replicator configuration files on disk.

      Issues: CT-761

      For more information, see The tungsten_post_process Command (in [Tungsten Clustering (for MySQL) 6.1 Manual]).

  • Tungsten Connector

    • The Tungsten Stack now supports the new MySQL 8.0 authentication plugins. Both sha256_password and caching_sha2_password (the new default) are supported by the Replicator, Manager and Connector.

      More info on these authentication plugins can be found here: https://dev.mysql.com/doc/refman/8.0/en/sha256-pluggable-authentication.html

      The Drizzle driver has been updated to support these new authentication methods, and the MySQL Connector/J 8 is also supported.

      In order to be fully transparent with the new defaults, when connected to a MySQL 8+ data source, the Connector will advertise caching_sha2_password as the default plugin.

      With earlier versions of MySQL (pre-8.0), the previous default mysql_native_password is used by default and advertised to the client applications.

      In order to override the default behavior, a new Connector property option for tpm (in [Tungsten Clustering (for MySQL) 6.1 Manual]), property=defaultAuthPlugin={autodetect|caching_sha2_password|mysql_native_password}, and is set to autodetect by default.

      Note that if property=defaultAuthPlugin is set to caching_sha2_password, the sha256_password authentication is automatically also supported.

      Warning

      Please note that the Connector does not support public key retrieval as of yet.

      Also note that, for backwards compatibility, the Connector forces the “CLIENT_DEPRECATE_EOF” to false, disallowing the usage of client session tracking requests (https://dev.mysql.com/doc/refman/5.7/en/session-state-tracking.html)

      Issues: CT-771

    • When logging is set to debug or trace, the Connector will print individual queries. In the past, advanced logging limited the display size of requests to 256 characters to prevent overwhelming the logs in terms of both space and filesystem I/O.

      Some customers need to display more than that, so it is now possible to adjust the size of the statements displayed in debug or trace logging modes. This is handled by a new Connector property option for tpm (in [Tungsten Clustering (for MySQL) 6.1 Manual]), property=statement.display.size.in.kb=NNN, which is defined as the maximum query length to display in Kbytes, and now defaults to 1KB.

      Warning

      Warning: setting this option to a high value while DEBUG or TRACE is enabled will quickly fill logs and disk, in addition to using up disk I/O's!

      For example, if the raw query size is 4KB, then a setting of 1KB would simply display the first 1024 bytes of the query and truncate/discard the rest from a logging perspective.

      For more information about configuring debug and trace logging, please visit Generating Advanced Diagnostic Information (in [Tungsten Clustering (for MySQL) 6.1 Manual]).

      Issues: CT-990

Bug Fixes

  • Tungsten Connector

    • OLD BEHAVIOR: If the Master data source was not accessible when the Connector started (i.e. connection refused, etc.), the connector would still fully initialize, leading to a running Connector without an accessible data source. This has the side effects of having default configuration values for both wait_timeout and server_version instead of properly auto-detected values based on the MySQL server settings.

      NEW BEHAVIOR: The Connector will now wait indefinitely for a Master to become available before finishing startup.

      Issues: CT-930

    • Introduced a new tpm (in [Tungsten Clustering (for MySQL) 6.1 Manual]) flag allowing for tuning Connector thread stack size, which can be required in particular cases where large requests are sent as text to a connector configured for automated read/write splitting (smartscale and direct reads).

      The setting is commented out by default, leaving the JVM use its own default, generally 1024.

      Setting tpm (in [Tungsten Clustering (for MySQL) 6.1 Manual]) option connector-thread-stack-size={value in kb} will override this value.

      Note

      Please note that since the new size will be allocated for each incoming connection, increasing the thread stack size will affect the total runtime memory used by the connector instance

      Issues: CT-973

Tungsten Clustering 6.1.0 Includes the following changes made in Tungsten Replicator 6.1.0

Release 6.1.0 contains both significant improvements as well as some needed bugfixes. One of the main features of this release is MySQL 8 support.

Improvements, new features and functionality

  • Command-line Tools

    • Two new utility scripts have been added to the release to help with setting the Replicator position:

      - tungsten_find_position, which assists with locating information in the THL based on the provided MySQL binary log event position and outputs a dsctl set (in [Tungsten Replicator 6.1 Manual]) command as output.

      - tungsten_find_seqno, which assists with locating information in the THL based on the provided sequence number and outputs a dsctl set (in [Tungsten Replicator 6.1 Manual]) command as output.

      Issues: CT-934

  • Core Replicator

    • A new, beta-quality command has been included called prov-sl.sh which is intended to eventually replace the current tungsten_provision_slave (in [Tungsten Replicator 6.1 Manual]) script.

      Currently, prov-sl.sh supports provisioning slaves using mysqldump and xtrabackup tools, and is MySQL 8-compatible. 

      The prov-sl.sh command is written in Bash, has less dependencies compared to the current version and is meant to fix a number of issues with the current version.

      Backups are streamed from source to target so that an intermediate write to disk is not performed, resulting in faster provisioning times.

      Logs are written to $CONTINUENT_ROOT/service_logs/prov-sl.log (i.e. /opt/continuent/service_logs/prov-sl.log).

      For example, provision a slave from [source db] using mysqldump (default):

      shell> prov-sl.sh -s {source db}

      As another example, use xtrabackup for the backup method, with 10 parallel threads (default is 4), and ssh is listening on port 2222:

      shell> prov-sl.sh -s {source db} -n xtrabackup -t 10 -p 2222

      Warning

      At the moment, prov-sl.sh does not support MultiMaster topologies when used with Tungsten Clustering, however it will be included in a future release.

      Issues: CT-614, CT-723, CT-809, CT-855, CT-963

    • Upgraded the Drizzle driver to support MySQL 8 authentication protocols (SHA256, caching_sha2).

      Issues: CT-914, CT-931, CT-966

    • The Redshift Applier now allows AWS authentication using IAM Roles. Previously authentication was possible via Access and Secret Key pairs only.

      Issues: CT-980

      For more information, see Redshift Preparation for Amazon Redshift Deployments (in [Tungsten Replicator 6.1 Manual]).

Bug Fixes

  • Command-line Tools

    • When executing mysqldump, all Tungsten tools no longer use the --add-drop-database flag as it will prevent MySQL 8+ from restoring the dump.

      Issues: CT-935

    • Fixed a bug where tpm diag (in [Tungsten Replicator 6.1 Manual]) would generate an empty zip file if the hostnames contain hyphens (-) or periods (.)

      Issues: CT-1032

  • Core Replicator

    • Added support for missing charset GB18030 to correct WARN extractor.mysql.MysqlBinlog Unknown charset errors.

      Issues: CT-915, CT-932

    • Loading data into Redshift would fail with the following error if a row of data contained a specific control character (0x00 (NULL))

      Missing newline: Unexpected character 0x30 found at location nnn

      Issues: CT-984

    • Now properly extracting the Geometry datatype.

      Issues: CT-997

    • The ddl_map.json file used by the apply_schema_changes filter was missing a rule to handle ALTER TABLE statements when replicating between MySQL and Redshift

      Issues: CT-1002

    • The extract_schema_change filter wasn't escaping " (double-quotes) and the generated JSON would then cause the applier to error with

      pendingExceptionMessage: SyntaxError: missing } after property list »
      (../../tungsten-replicator//support/filters-javascript/apply_schema_changes.js#236(eval)#1)

      Issues: CT-1011