Release Notes

Continuent Ltd

Abstract

This document provides release notes for all (active) released versions of Continuent software.

Build date: 2024-02-23 (becc38cf)

Up to date builds of this document: Release Notes (Online), Release Notes (PDF)


Table of Contents

1. Tungsten Clustering Release Notes
1.1. Tungsten Clustering 7.1.1 GA (15 Dec 2023)
1.2. Tungsten Clustering 7.1.0 GA (16 Aug 2023)
1.3. Tungsten Clustering 7.0.3 GA (4 Apr 2023)
1.4. Tungsten Clustering 7.0.2 GA (9 Dec 2022)
1.5. Tungsten Clustering 7.0.1 GA (13 June 2022)
1.6. Tungsten Clustering 7.0.0 GA (29 Mar 2022)
1.7. Tungsten Clustering 6.1.24 GA (11 Dec 2023)
1.8. Tungsten Clustering 6.1.23 GA (31 Aug 2023)
1.9. Tungsten Clustering 6.1.22 GA (6 July 2023)
1.10. Tungsten Clustering 6.1.21 GA (18 Apr 2023)
1.11. Tungsten Clustering 6.1.20 GA (19 Dec 2022)
1.12. Tungsten Clustering 6.1.19 GA (17 Oct 2022)
1.13. Tungsten Clustering 6.1.18 GA (7 Sept 2022)
1.14. Tungsten Clustering 6.1.17 GA (16 May 2022)
1.15. Tungsten Clustering 6.1.16 GA (20 Dec 2021)
1.16. Tungsten Clustering 6.1.15 GA (19 Oct 2021)
1.17. Tungsten Clustering 6.1.14 GA (17 Aug 2021)
1.18. Tungsten Clustering 6.1.13 GA (27 May 2021)
1.19. Tungsten Clustering 6.1.12 GA (8 Mar 2021)
1.20. Tungsten Clustering 6.1.11 GA (21 Jan 2021)
1.21. Tungsten Clustering 6.1.10 GA (15 Dec 2020)
1.22. Tungsten Clustering 6.1.9 GA (23 Nov 2020)
1.23. Tungsten Clustering 6.1.8 GA (2 Nov 2020)
1.24. Tungsten Clustering 6.1.7 GA (5 Oct 2020)
1.25. Tungsten Clustering 6.1.6 GA (20 Aug 2020)
1.26. Tungsten Clustering 6.1.5 GA (5 Aug 2020)
1.27. Tungsten Clustering 6.1.4 GA (4 June 2020)
1.28. Tungsten Clustering 6.1.3 GA (17 February 2020)
1.29. Tungsten Clustering 6.1.2 GA (20 January 2020)
1.30. Tungsten Clustering 6.1.1 GA (28 October 2019)
1.31. Tungsten Clustering 6.1.0 GA (31 July 2019)
2. Tungsten Replicator Release Notes
2.1. Tungsten Replicator 7.1.1 GA (15 Dec 2023)
2.2. Tungsten Replicator 7.1.0 GA (16 Aug 2023)
2.3. Tungsten Replicator 7.0.3 GA (4 Apr 2023)
2.4. Tungsten Replicator 7.0.2 GA (19 Dec 2022)
2.5. Tungsten Replicator 7.0.1 GA (13 June 2022)
2.6. Tungsten Replicator 7.0.0 GA (29 Mar 2022)
2.7. Tungsten Replicator 6.1.24 GA (11 Dec 2023)
2.8. Tungsten Replicator 6.1.23 GA (31 Aug 2023)
2.9. Tungsten Replicator 6.1.22 GA (6 July 2023)
2.10. Tungsten Replicator 6.1.21 GA (18 Apr 2023)
2.11. Tungsten Replicator 6.1.20 GA (19 Dec 2022)
2.12. Tungsten Replicator 6.1.19 GA (17 Oct 2022)
2.13. Tungsten Replicator 6.1.18 GA (7 Sept 2022)
2.14. Tungsten Replicator 6.1.17 GA (16 May 2022)
2.15. Tungsten Replicator 6.1.16 GA (20 Dec 2021)
2.16. Tungsten Replicator 6.1.15 GA (19 Oct 2021)
2.17. Tungsten Replicator 6.1.14 GA (17 Aug 2021)
2.18. Tungsten Replicator 6.1.13 GA (27 May 2021)
2.19. Tungsten Replicator 6.1.12 GA (8 Mar 2021)
2.20. Tungsten Replicator 6.1.11 GA (21 Jan 2021)
2.21. Tungsten Replicator 6.1.10 GA (15 Dec 2020)
2.22. Tungsten Replicator 6.1.9 GA (23 Nov 2020)
2.23. Tungsten Replicator 6.1.8 GA (2 Nov 2020)
2.24. Tungsten Replicator 6.1.7 GA (5 Oct 2020)
2.25. Tungsten Replicator 6.1.6 GA (20 Aug 2020)
2.26. Tungsten Replicator 6.1.5 GA (5 Aug 2020)
2.27. Tungsten Replicator 6.1.4 GA (4 June 2020)
2.28. Tungsten Replicator 6.1.3 GA (17 February 2020)
2.29. Tungsten Replicator 6.1.2 GA (20 January 2020)
2.30. Tungsten Replicator 6.1.1 GA (28 October 2019)
2.31. Tungsten Replicator 6.1.0 GA (31 July 2019)
3. Tungsten Dashboard Release Notes
3.1. Tungsten Dashboard 1.0.15 GA (14 February 2024)
3.2. Tungsten Dashboard 1.0.14 GA (11 April 2023)
3.3. Tungsten Dashboard 1.0.13 GA (31 January 2023)
3.4. Tungsten Dashboard 1.0.12 GA (14 December 2022)
3.5. Tungsten Dashboard 1.0.11 GA (8 November 2022)
3.6. Tungsten Dashboard 1.0.10 GA (7 March 2022)
3.7. Tungsten Dashboard 1.0.9 GA (12 August 2020)
3.8. Tungsten Dashboard 1.0.8 GA (4 June 2020)
3.9. Tungsten Dashboard 1.0.7 GA (26 November 2019)
3.10. Tungsten Dashboard 1.0.6 GA (3 September 2019)
3.11. Tungsten Dashboard 1.0.5 GA (28 June 2019)
3.12. Tungsten Dashboard 1.0.4 GA (11 April 2019)
3.13. Tungsten Dashboard 1.0.3 GA (22 March 2019)
3.14. Tungsten Dashboard 1.0.2 GA (20 September 2018)
3.15. Tungsten Dashboard 1.0.1 GA (17 September 2018)
3.16. Tungsten Dashboard 1.0.0 GA (10 May 2018)
4. Tungsten AMI Release Notes
4.1. Tungsten AMI 5.0.0 GA (NYR)
4.2. Tungsten AMI 4.0.0 GA (17 December 2020)
4.3. Tungsten AMI 3.0.0 GA (20 July 2020)
4.4. Tungsten AMI 2.0.0 GA (11 November 2019)
4.5. Tungsten AMI 1.0.0 GA (6 June 2019)

1. Tungsten Clustering Release Notes

1.1. Tungsten Clustering 7.1.1 GA (15 Dec 2023)
1.2. Tungsten Clustering 7.1.0 GA (16 Aug 2023)
1.3. Tungsten Clustering 7.0.3 GA (4 Apr 2023)
1.4. Tungsten Clustering 7.0.2 GA (9 Dec 2022)
1.5. Tungsten Clustering 7.0.1 GA (13 June 2022)
1.6. Tungsten Clustering 7.0.0 GA (29 Mar 2022)
1.7. Tungsten Clustering 6.1.24 GA (11 Dec 2023)
1.8. Tungsten Clustering 6.1.23 GA (31 Aug 2023)
1.9. Tungsten Clustering 6.1.22 GA (6 July 2023)
1.10. Tungsten Clustering 6.1.21 GA (18 Apr 2023)
1.11. Tungsten Clustering 6.1.20 GA (19 Dec 2022)
1.12. Tungsten Clustering 6.1.19 GA (17 Oct 2022)
1.13. Tungsten Clustering 6.1.18 GA (7 Sept 2022)
1.14. Tungsten Clustering 6.1.17 GA (16 May 2022)
1.15. Tungsten Clustering 6.1.16 GA (20 Dec 2021)
1.16. Tungsten Clustering 6.1.15 GA (19 Oct 2021)
1.17. Tungsten Clustering 6.1.14 GA (17 Aug 2021)
1.18. Tungsten Clustering 6.1.13 GA (27 May 2021)
1.19. Tungsten Clustering 6.1.12 GA (8 Mar 2021)
1.20. Tungsten Clustering 6.1.11 GA (21 Jan 2021)
1.21. Tungsten Clustering 6.1.10 GA (15 Dec 2020)
1.22. Tungsten Clustering 6.1.9 GA (23 Nov 2020)
1.23. Tungsten Clustering 6.1.8 GA (2 Nov 2020)
1.24. Tungsten Clustering 6.1.7 GA (5 Oct 2020)
1.25. Tungsten Clustering 6.1.6 GA (20 Aug 2020)
1.26. Tungsten Clustering 6.1.5 GA (5 Aug 2020)
1.27. Tungsten Clustering 6.1.4 GA (4 June 2020)
1.28. Tungsten Clustering 6.1.3 GA (17 February 2020)
1.29. Tungsten Clustering 6.1.2 GA (20 January 2020)
1.30. Tungsten Clustering 6.1.1 GA (28 October 2019)
1.31. Tungsten Clustering 6.1.0 GA (31 July 2019)

1.1. Tungsten Clustering 7.1.1 GA (15 Dec 2023)

Version End of Life. Not Yet Set

Release 7.1.1 contains a number of key bug fixess including one majore bug fix that prevents switches from working correclty after upgraging from v6.

Behavior Changes

The following changes have been made to Tungsten Cluster and may affect existing scripts and integration tools. Any scripts or environment which make use of these tools should check and update for the new configuration:

  • Command-line Tools

    • The tpm diag command now collects ps -aux output in addition to ps -efl when possible.

      Issues: CT-2119

  • Tungsten Connector

    • Changed default value of mgr-policy-fence-slave setting so that a replica data source is put offline 60s after its replicator goes offline. This change will prevent routing traffic to replicas that are stale.

      Issues: CT-2096

Improvements, new features and functionality

  • Command-line Tools

    • The tapi command now supports the ?no-connectors=1 argument in APIv2 via the new cli argument --no-connectors

      Issues: CT-2174

    • The tpm mysql command now accepts the --force argument to function on Witness nodes that have a running database.

      Issues: CT-2180

Bug Fixes

  • Installation and Deployment

    • Installations will now succeed on hosts running Ruby 3.x

      Issues: CT-2196

  • Command-line Tools

    • The tpm report command now correctly displays security info tpm options as blank when none exists.

      Issues: CT-2176

    • The tpm update command no longer fails to display foreign-owned dot-files and directories.

      Issues: CT-2190

    • The tpm mysql command now properly accepts the -e argument as a shortcut for the --execute mysql cli client argument.

      Issues: CT-2217

    • Fixes a regression in tpm cert that prevented BASE_DIR in tungsten.env from being used properly.

      Issues: CT-2237

  • Tungsten Manager

    • Fixed a thread synchronization issue that can cause memory leaks.

      Issues: CT-2197

    • v6 API configuration properties within the INI file, are now also recognised by v7.

      Issues: CT-2231

    • Fixed a bug when upgrading from 6.x to 7.x that prevented switches/failovers from working. Also the change of datasource-group-id in the INI will now be correctly reflected after a tpm update.

      Issues: CT-2236

1.2. Tungsten Clustering 7.1.0 GA (16 Aug 2023)

Version End of Life. Not Yet Set

Release 7.1.0 is the next major v7 release containing a number of important bug fixes and key new features.

Behavior Changes

The following changes have been made to Tungsten Cluster and may affect existing scripts and integration tools. Any scripts or environment which make use of these tools should check and update for the new configuration:

  • Command-line Tools

    • No longer using connector graceful-stop together with systemd for upgrades. The underlying change of binary confuses systemd scripts.

      Issues: CT-2113

    • cctrl now accepts services names with capital letters, dots and hyphens

      Issues: CT-2163

    • The tpm copy-keys command has been renamed to tpm copy and a new command has been created tpm cert copy with the same functionality.

      Issues: CT-2186

  • Backup and Restore

    • When performing a provision via rsync, tprovision will now sleep for 2 seconds after locking tables to make sure all transactions have finished writing to disk.

      Issues: CT-2169

Improvements, new features and functionality

  • Behavior Changes

    • Additional logging will now be added to the replicator logs during switchover/failover operations to enable better debugging in the event of issues.

      Issues: CT-1448

  • Installation and Deployment

    • Running tpm uninstall will now save all of the Tungsten database tracking schemas for later use. There is also a new tpm keep command, which allows the tracking schemas to be saved to disk at any time in multiple formats (.json, .dmp and .cmd)

      Issues: CT-2131

    • Added tpm flag deploy-systemd as a more meaningful alias to install

      Issues: CT-2152

  • Command-line Tools

    • Added a new option --preserve-schema to the tpm uninstall command in order to leave the tracking schema in the database.

      Issues: CT-561

    • A new command tpm cert has been added to aid in the creation, rotation and management of certificates for all areas of Tungsten.

      Known limitations: Percona 5.6, and all versions of MariaDB do not provide the mysql_ssl_rsa_setup command required by tpm cert gen mysqlcerts.

      Issues: CT-2085

    • The tpm report command now displays the security-specific information for each channel shown, including file paths and tpm options. Security information and tpm options for each channel will also be shown when --extra is used with --list.

      Also added the tpm ask certs command with expiry and sha2566 info per alias, along with the tpm ask certtpm and tpm ask certlocations commands with reference information about the security.properties file.

      Added help text for tpm ask --long which shows the key/variable name along with the value. The tmonitor -t test command now shows only the actual Tungsten-specific metrics lines.

      Added tmonitor -T test to show metrics help and headers along with actual tungsten_* metrics lines.

      Issues: CT-2088

    • tungsten show processlist could error with a disconnection message listing connections disconnected on the mysql server side.

      Issues: CT-2112

    • The tpm diag command now captures the output of cctrl> cluster topology validate

      Issues: CT-2115

    • The tpm diag command now supports the --skipsudo and --nosudo arguments to prevent operations from using the sudo command. Using this option may result in tpm diag skipping/failing various gathers due to a lack of access.

      Issues: CT-2146

    • The tungsten_send_diag command has a new argument --all which will tell the tpm diag command to gather all hosts with -a, and this replaces the previous method for gathering all hosts for tungsten_send_diag, --args '--all'

      Issues: CT-2150

  • Backup and Restore

    • MySQL clone can now be used as an option for recovery using tprovision.

      Issues: CT-1417

    • tungsten_get_mysql_datadir can now return additional mysql database directories.

      Issues: CT-1985

    • tprovision will now add a heartbeat after restore. This will ensure the replicator can be put back online when there is no load on the cluster.

      Issues: CT-2005

    • The tprovision script will sleep for 5 seconds by default when using the rsync method after issuing a flush logs. The sleep value is configurable as a command line option --flush-after-sleep.

      Issues: CT-2101

  • Filters

    • A new shardbyrules filter has been added that will allow rule based sharding of replication based on user confgurable rules that would allow sharding at table level, whereas previoulsy sharding would only be handled at schema level.

      For more information, see shardbyrules Filter Documentation

      Issues: CT-2164

  • Tungsten Connector

    • Tungsten Connector now supports dual passwords, mirroring the MySQL v8.0.14+ functionality. When changing a user password, the previous password can be retained as long as needed in order to allow changing account passwords with no downtime.

      Issues: CT-2127

    • Added a flag to avoid generating and sending EOF packets to client applications when CLIENT_DEPRECATE_EOF is not set on both client and mysql server sides. This fixes an issue with Go-MySQL-Driver and prepared statements.

      When using Go MySQL Driver, flag --connector-generate-eof=false should be specified. Default is set to true for backwards compatibility.

      Issues: CT-2177

  • Core Clustering

    • Upgraded JGroups library to 4.2.22

      Issues: CT-2011

    • There is a new datasource-group-id TPM option. In a single cluster the nodes with the same datasource-group-id will form a Distributed Datasource Group (DDG).

      Issues: CT-2051

  • Monitoring

    • Prometheus exporters now provide the ssl cert expiration date as an epoch value in addition to the label.

      Issues: CT-2099

    • Added Prometheus exporter metrics for composite parent and sub-services.

      Issues: CT-2121

    • Prometheus libraries have been upgraded from version 0.8.1 to 0.16.0

      Issues: CT-2166

    • The ability to configure the Java Virtual Machine (JVM) settings for the manager has been made easier, by the use of the manager_java_settings.conf file. For more information see Adjusting JVM Settings for the Manager

      Issues: CT-2167

Bug Fixes

  • Installation and Deployment

    • Fixed RPM package script to run tpm install instead of tpm update when installing the rpm

      Issues: CT-2130

  • Command-line Tools

    • The tpm command now handles situations when the Manager process is not running.

      Issues: CT-2103

    • tpm uninstall would sometimes print "ERROR >> db1 >> undefined method '+' for nil:NilClass"

      Issues: CT-2104

    • The tungsten_find_orphaned command now handles some edge cases more gracefully.

      Issues: CT-2107

    • The cctrl cluster topology validate command now checks, in a CAA setup, if the relay in the subservice is on the same host as the primary, and reports if there is a mismatch.

      Issues: CT-2114

    • The tpm command now searches more places to locate shell commands that are called, especially useful when $CONTINUENT_ROOT/share/env.sh is not sourced.

      Issues: CT-2182

  • Backup and Restore

    • tprovision would produce errors if the local hostname were different from the hostname used in the Tungsten install (short vs long names).

      Issues: CT-1363

    • tprovision will now print an error message and exit if the MySQL datadir does not exit.

      Issues: CT-1901

    • tprovision would accept bogus options and not produce an error. This has now been fixed.

      Issues: CT-2045

    • tprovision will now timeout if ssh is blocked from the target to the source host.

      Issues: CT-2139

    • Using mysqldump for tprovision could incorrectly create a new SSL key pair.

      Issues: CT-2142

  • Core Replicator

    • Improved a query that is run by Tungsten when fetching tables metadata (column names, datatypes, etc). While it is not generally needed, the unoptimized query can run badly (especially) against old mysql versions with a lot of databases / tables. For now, the new optimized query is not used by default, but this could change in some future version.

      This can be enabled by using the following property :

      property=replicator.datasource.global.connectionSpec.usingOptimizedMetadataQuery=true

      Issues: CT-2077

    • Fixed an issue while processing geometry data with SRID 4326 that would swap longitude and latitude. This applies only to MySQL 8, as prior MySQL versions do not allow specifying the order when applying a WKB (Well-known binary) to MySQL

      Issues: CT-2172

  • Tungsten Connector

    • Fixed NullPointerExceptions when reading packets from disconnected client connections.

      Issues: CT-2132

    • Fixed a log4j configuration issue where the connector-audit.log and connector-api.log files that have been rotated will end up in cluster-home/bin/directory instead of tungsten-connector/log/.

      Issues: CT-2137

    • Fixed an issue with topology validation when running tpm promote-connector on a witness host

      Issues: CT-2161

    • Fixed an issue where a failover could hang in rare cases when security and Proxy mode are enabled. SSL connections to a failed data source could enter a deadlock and block failover for up to 15 minutes.

      Note

      This issue only affects users running Java 11, and is related to https://bugs.openjdk.org/browse/JDK-8241239

      Issues: CT-2183, CT-2187

  • Core Clustering

    • In a Composite Active/Active topology, issuing SHUN/DRAIN or WELCOME on a node in cctrl would only affect the node in the main cluster service. This action will now also be applied to the same node within the x_from_y sub-service.

      Issues: CT-2145

  • Tungsten Manager

    • A set of changes to improve cctrl responsiveness

      • cctrl now properly lists responsive connectors even if some fail to return their status in a timely fashion (for example with slow networks)

      • Under the above condition, cctrl will return in 5-6 seconds rather than in up to 30 seconds when there is network congestion/partition.

      • Under normal conditions, cctrl responds significantly more quickly - up to 3x faster - due to optimized communications between cctrl and remote connectors.

      • cctrl commands are no longer slowed down/blocked by internal ping traffic to connectors.

      • Fixed an issue where individual connectors cannot be addressed in 'router' commands.

      Issues: CT-1795

1.3. Tungsten Clustering 7.0.3 GA (4 Apr 2023)

Version End of Life. Not Yet Set

Release 7.0.3 contains a number of key bug fixes and improvements.

Behavior Changes

The following changes have been made to Tungsten Cluster and may affect existing scripts and integration tools. Any scripts or environment which make use of these tools should check and update for the new configuration:

  • Installation and Deployment

    • Added LimitNOFILE=infinity in systemd scripts to workaround an issue with open files limit for clusters installed with systemd

      Issues: CT-2023

  • Command-line Tools

    • Deprecated tungsten_get_position and tungsten_set_position scripts have been removed.

      Issues: CT-638

    • The tpm ssh-copy-cert command has been deprecated and removed in this release.

      Issues: CT-828

    • The tps.pl backup tool now shows the available command-line arguments when the TAB key is pressed.

      Issues: CT-1246

    • The tungsten_skip_all command (along with alias tungsten_skip_seqno) now shows the full pendingExceptionMessage instead of just pendingError, and the More choice shows the pendingErrorEventId and the pendingError.

      Issues: CT-2030

    • Enabling a custom backup script does not require it to be made the default option anymore, i.e. Setting --repl-backup-method to the custom backup script is not required anymore.

      Issues: CT-2039

    • The tungsten_find_seqno command is now a wrapper for tpm find-seqno.

      Issues: CT-2059

    • The tpm command now prints a warning when running as the root OS user during operations that make changes.

      Issues: CT-2064

    • The tpm report command with with --extra now shows the associated Connector listener ports and mode per port for when the connector mode is "listeners".

      Issues: CT-2087

  • Tungsten Manager

    • Increased the default manager liveness heartbeat interval from 1s to 3s.

      Issues: CT-2095

Improvements, new features and functionality

  • Command-line Tools

    • A new thl tail command has been added, allowing you to view the live THL changes as they are generated.

      Issues: CT-405

    • A new command, tpm copy-keys, has been added to assist with the distribution of security-related files after an INI-based installation. This tool requires ssh access between the cluster nodes for the copy to work.

      Issues: CT-1910

    • The tungsten_get_ports command now supports the /opt/continuent/tungsten/tungsten-connector/conf/interfaces.json file, if found.

      Issues: CT-2003

    • The tungsten_get_ports command now use values based on values obtained from tpm query values when possible for the greatest output accuracy.

      Issues: CT-2008

    • The tpm ask summary command has two new keys, isSmartScale and isDirect, which are also available individually on the command line.

      Issues: CT-2035

    • When in Connector Proxy mode, the tpm report command now displays the Direct and SmartScale status via Channel 2: Connector to Database

      Issues: CT-2037

    • Shell command tab completion for Tungsten-specific commands has been updated to support recent additions and changes.

      Issues: CT-2057

  • Core Replicator

    • THL transfer protocol can now be changed dynamically. This can be done using the trepctl command tool

      shell> trepctl setdynamic -property replicator.thl.protocol.client.serialization -value DEFLATE 
      Property replicator.thl.protocol.client.serialization successfully set to DEFLATE

      This can also be done through API v2

      curl --user tungsten:secret --insecure --request POST 'https://127.0.0.1:8097/api/v2/replicator/service/alpha/thl/inflight-protocol' -d 
      '{
       "payloadType":"StringPayload",
       "payloadVersion":"1",
       "payload":
       {
       "value":"legacy"
       }
      }' 
      
      { 
       "payloadType": "StringPayload", 
       "payloadVersion": "1", 
       "payload": 
       { 
       "value": "THL inflight protocol is now set to legacy" 
       } 
      }

      Issues: CT-1862

  • Tungsten Connector

    • Improved smart scale handling of session and global changes so they're all played against connected replica or in case of reconnection. As an example, SET SESSION TRANSACTION READ ONLY will be sent to both primary and replica connections so that a subsequent SELECT @@SESSION.transaction_isolation will return the expected value, even if executed on a replica.

      Issues: CT-1600

    • Added user-based authentication to existing host-based authentication to restrict connections per host and per user.

      For more details, see Host-based Authentication

      Issues: CT-1978

    • A new advanced listeners feature has been added to the connector for even great control of client connectivity.

      For more details, see Advanced Listeners

      Issues: CT-2000

Bug Fixes

  • Command-line Tools

    • Fixed rsync availability check in TPM to error out early when the executable is not found in the path

      Issues: CT-1028

    • The tungsten_get_status command now handles shunned nodes properly.

      Issues: CT-2026

    • The tpm ask keys command now lists all available keys properly.

      Issues: CT-2041

    • The tpm update command no longer aborts with a Dataservice was NOT removed error when service names with a hyphen are defined in the configuration. We strongly discourage the use of hyphens (-) in service names and recommend underscores (_) instead.

      Issues: CT-2052

    • The tpm validate-update command now performs the delete-service checks properly.

      Issues: CT-2054

    • The tungsten_find_events command no longer fails with error Undefined subroutine &main::info

      Issues: CT-2058

    • tpm now builds affinity the same way in bridge and proxy modes.

      Issues: CT-2060

    • Fixes an issue when tprovision would delete the MySQL PEM files in the datadir.

      Issues: CT-2067

    • The tpm diag command will now locate and kill any remaining tpm diag child processes on the calling host.

      Issues: CT-2068

    • Suppressed wrong tpm errors with autodetect hosts.

      Issues: CT-2072

    • Fixed a tpm issue trying to restore an empty policy.

      Issues: CT-2074

  • Core Replicator

    • Fixed an issue that was occurring when an ALTER USER is applied on a mysql instance that uses lower case identifiers.

      Issues: CT-2024

    • Fixed an issue where an extracted JSON field could be invalid, with doubled commas in JSON arrays.

      Issues: CT-2049

    • Fixed an issue where status would fail if issued while a service was reconfiguring (in the middle of a setrole command for example).

      Issues: CT-2081

  • Tungsten Connector

    • Fixed an issue with CLIENT_QUERY_ATTRIBUTES and/or CLIENT_DEPRECATE_EOF flags that would prevent fallback bridge mode to handle proxy connections properly.

      Issues: CT-2020

    • Fixed an issue in Dynamic Active-Active environments where smartScale would always read from the site with write affinity, even if a different site is specified for read-affinity.

      Issues: CT-2040

1.4. Tungsten Clustering 7.0.2 GA (9 Dec 2022)

Version End of Life. Not Yet Set

Release 7.0.2 contains a number of key bug fixes and improvements.

Behavior Changes

The following changes have been made to Tungsten Cluster and may affect existing scripts and integration tools. Any scripts or environment which make use of these tools should check and update for the new configuration:

  • Command-line Tools

    • The various user-xxxx.log files are no longer generated

      Issues: CT-1914

    • The check_tungsten.sh was deprecated in release 6.1.18 and has now been removed from this release.

      Issues: CT-1939

    • Both the tungsten_get_ports and tpm report commands have been updated to use the ss OS command when the netstat OS command is unavailable or deprecated.

      Issues: CT-2007

  • Core Replicator

    • repl_svc_extractor_multi_frag_service_detection is now turned ON by default. Event shards are determined at extraction time. With fragmented events, the shard cannot be determined by only reading the first fragment, but needs to check the last fragment as well. With this setting turned OFF, there is no issue with pipelines that don't need it, i.e. no parallel apply downstream replicas. However, as this is done at extract time, THL contains this information, and adding or changing a replica using parallel apply could introduce issues.

      Note

      It can be disabled if you see a performance overhead but this should be done with caution. For Aurora<>Aurora Active/Active deployments it is essential that this property be left ON.

      Issues: CT-1959

  • Tungsten Connector

    • Improved tungsten show processlist by running underlying commands in parallel.

      Issues: CT-1569

  • Core Clustering

    • Changed the behavior of the connector and manager upon local site failover:

      No longer sending write traffic to the remote site unless the local site is fully offline. In case of local failover, the connector will now pause connections until a new primary is elected. This will avoid risks of out-of-order apply after local failover

      Issues: CT-2009

  • Tungsten Manager

    • A failsafe shunned cluster (Caused by a network split) will be auto recovered after the network connection is re-established.

      Issues: CT-241

Improvements, new features and functionality

  • Command-line Tools

    • rsync is now an option in tprovision, in addition to using xtrabackup and mysqldump. To use rsync, specify -m rsync.

      Using rsync by default will provision a replica in 2 passes:

      • The first pass will live copy (seed) the replica from the source.

      • The second pass will quiesce the source and run the rsync again, resulting in shorter down time than a single pass rsync

      Issues: CT-338

    • A new -c option is now available with some trepctl commands that can be used in conjunction with the -r option to indicate the number of times to refresh before automatically terminating. For example, the following command:

      shell> trepctl perf -r 3 -c 10

      Will refresh the output every 3 seconds, 10 times.

      Issues: CT-679

    • The trepctl status will now show the last known applied seqno and latency.

      This information is stored on disk at regular intervals (10s minimum) so as not to overload the replicator, therefore the value can be shown as slightly old dependant on when the status command was issued.

      By default, this feature is disabled. It can be enabled by setting the following parameter in the configuration :

      svc-applier-last-applied-write-interval=20

      This will write current position to disk every 20 seconds. This information is also exported by the Prometheus exporter.

      If the service is online, it will display the current value (the same as appliedLastSeqno and appliedLatency)

      shell> trepctl status 
      Processing status command... 
      NAME VALUE 
      ---- ----- 
      appliedLastEventId : mysql-bin.000017:0000000151329854;70 
      appliedLastSeqno : 999 
      appliedLatency : 347707.0 
      ... 
      lastKnownAppliedLatency: 347707.0 
      lastKnownAppliedSeqno : 999 
      ...

      Issues: CT-1823

    • The tungsten_merge_logs command now supports the --before TIMESTAMP and --after TIMESTAMP filters

      Issues: CT-1869

    • Added a new log file (tungsten-replicator/log/data-drift.log) for data drift messages, i.e. :

      • an update statement was logged on primary, but did not update any row on replica

      • a delete statement was logged on primary, but did not delete any row on replica

      Issues: CT-1873

    • The tpm ask summary command now provides the coordinator host and the isCoordinator boolean if the Manager is running on that node.

      Also, tpm ask now supports direct calls to coordinator, \{isCoordinator|iscoordinator} and \{isBridgeMode|isBridge|bridge|isbridge|isbridgemode}.

      Issues: CT-1874

    • The tungsten_generate_haproxy_for_api and tpm generate-haproxy-for-api commands now support using connector hosts in the backend definitions via -c, and extra backend flags to the backend hosts lines using -f.

      Issues: CT-1909

    • The tungsten_generate_haproxy_for_api and tpm generate-haproxy-for-api commands no longer call the Perl Data::Dumper module.

      Issues: CT-1915

    • The tungsten_reset_manager command now supports the ability to simply print out the path or paths to be cleared, one per line via the -l or --list arguments.

      Issues: CT-1917

    • The tmonitor command now accepts cli args to specify the ports and will auto-configure the ports if they have been changed via the Tungsten configuration.

      Issues: CT-1919

    • The tpm command calls to glob have been improved to be more strict and compliant.

      Issues: CT-1940

    • The tpm ask stages and tpm ask allstages commands have been added to display the Replicator stages for the current node (stages) and the stages for each role (allstages).

      Issues: CT-1943

    • The tpm ask command has five new variables available: dsrole & dsstate for the current datasource, and trrole & trstate for the current replicator, and nodeinfo which displays all 4 of the new variables.

      Issues: CT-1944

    • A new standalone status script has been added called tungsten_get_status that shows the datasources and replicators for all nodes in all services along with seqno and latency.

      Issues: CT-1962

    • The tpm report command now prints the hostname and listener ports where available when using the --extra|-x option or the new --ports option.

      Issues: CT-1969

    • A new dsctl option has been added to the thl command, and a new -event option added to thl list

      Full details and examples of these new options can be found here

      Issues: CT-2012

  • Core Replicator

    • Added a new feature that enables pausing a replicator stage for some amount of time.

      This will pause the given stage for 100 seconds.:

      trepctl pause -stage thl-to-q -time 100

      This will pause the stage indefinitely (or until restart, etc) Add -y to avoid the prompt message whether you are sure.

      trepctl pause -stage thl-to-q

      For the previous 2 commands, running a pause command again will override the previous command.

      This will resume the suspended stage (Note that if the stage is not paused, this will have no effect):

      trepctl resume -stage thl-to-q

      Note

      Please note this pause does not survive a replicator restart or a service offline/online.

      Issues: CT-1912

    • Per-service tuning of the replicator thl directory is now possible for multi-service replicator-only installs as well as for clustering. The given value should be the base directory, to which tungsten will add the service name. For example, the following entry in the tungsten.ini:

      [alpha]
      ...
      ...
      thl-directory=/drv1/thl
      ...

      Would result in the THL being placed in /drv1/thl/alpha

      Note

      Update of thl directory is only available when tpm is called from the staging installation directory, NOT from the running directory.

      Issues: CT-1927

    • A new replicator role (thl-applier) has been added to allow a replicator service to apply its locally available THL, without pulling from a remote host

      Issues: CT-1936

    • Added a way to configure the maximum number of rows that can be grouped together when applying row based events for multiple insert or delete statements.

      For these properties to be in effect, you must ensure that optimize-row-events=true is either explicitly set in your configuration, or not present (since it will be enabled by default)

      For example, the following settings will limit the number of inserted or deleted rows applied at once to 10:

      optimize-row-events-limit-insert-rows=10 
      optimize-row-events-limit-delete-rows=10

      The default values if not specified will be 50 for inserts and 100 for deletes. Note that for deletes to be optimized, the affected table MUST have a single column PK.

      Issues: CT-1980

  • Tungsten Connector

    • The connector graceful-stop command now supports systemd service manager properly. The connector stop command now takes an optional argument that will make it a graceful stop. If connector stop is run without the parameter, it will stop the connector immediately. If a positive number of seconds is passed, it will wait, at most, this timeout for connections to disconnect (refusing new connections), after which it will force close all connections and shutdown the connector. connector graceful-stop behavior is unchanged: without the parameter, the connector will wait "forever" for connections to disconnect. A positive timeout in seconds can be passed to sever connections after the given delay

      Issues: CT-1921

    • Two new commands have ben introduced to cctrl to provide better insight and control of the connectors. These are:

      • datasource <dsname> connections [-l]

        This command displays the current number of connections running to the given node through connectors.

      • datasource <dsname> drain [optional timeout]

        This command will prevent new connection to be made to the given data source, while ongoing connection remain untouched.

      For more detils on the usage and options, see cctrl datasource Command

      Issues: CT-1949

    • Added logging configuration example to print load balancers activity

      Issues: CT-1966

  • Tungsten Manager

    • Added a new option to TPM manager-replicator-offline-timeout=<timeout_in_sec> that configures the timeout for the manager to wait until the replicator goes offline. When parallel applier is in use the default timeout was too low, so it’s now user configurable so that it can be adjusted to suit different topologies. If not supplied, the default is 180 (3 minutes). This value should be sufficent in most use cases.

      Issues: CT-1892

  • API

    • New logs files have been added for the REST API, these are as follows:

      • service_logs/connector-api.log

      • service_logs/manager-api.log

      • service_logs/replicator-api.log

      Issues: CT-1983

Bug Fixes

  • Installation and Deployment

    • ddlscan, dsctl and tungsten_send_diag are now added to the aliases.sh script.

      Issues: CT-813

    • Fixes issues where fixed properties and filters passed to tpm in service stanzas were not being configured correctly

      Issues: CT-1463

    • No longer using Tanuki wrapper functionality to print jvm version, which was creating defunct java processes at startup, now using internal code.

      Issues: CT-1876

    • The tpm install and tpm update commands now properly support the --thl-ports option for cross-site subservices.

      Issues: CT-1953

  • Command-line Tools

    • The tungsten_skip_seqno command no longer fails when -i is specified, and now properly filters using --filter when there is a long error message.

      Issues: CT-1877

    • The tpm command now allows any case for section entries (i.e. [alpha_FROM_beta]) in the INI files.

      Issues: CT-1879

    • The tpm diag command now passes when the nodename defined in the tungsten.ini is the shortname, and DNS returns the FQDN.

      Issues: CT-1908

    • The tpm diag command now gathers the mysql.log file when SSL is enabled in the server.

      Issues: CT-1920

    • Note

      This fix was released in Tungsten Clustering and Tungsten Replicator 7.0.2 Build 161.

      The tpm command checks for the existence of the mysql command-line client when installing/upgrading. The process will no longer abort with an error on non-MySQL targets such as heterogeneous replicator appliers, or Active-Witness hosts.

      Issues: CT-1924, CT-2018

    • Fixes an issue that prevented dsctl from connecting to MySQL if SSL was enabled.

      Issues: CT-1928

    • The tpm mysql command will now gracefully handle being run on a non-database node.

      Issues: CT-1946

    • The tpm mysql command no longer aborts with an access denied error on CentOS 6.

      Issues: CT-1977

    • Both TungstenAPI and tpasswd now properly update .passwords.store.orig backup file so that proper manipulation of passwords won't trigger tpm update failure

      Issues: CT-1981

  • Backup and Restore

    • The cluster_backup script will no longer backup a replica if the replicator is in an ERROR state.

      Issues: CT-1036

  • Core Replicator

    • Fixes an issue that would prevent a service from going offline at a specified time (trepctl online -until-time) when parallel apply is enabled. This is a rework of CT-1243.

      Issues: CT-1684

    • Fixed a possible issue when recovering an old primary as a replica after failover when parallel apply is enabled, that could lead the replica to be unable to come online and require a reprovisioning of this replica.

      Issues: CT-1890

    • Fixes an issue that prevented geometry datatypes with SRID from being replicated.

      Issues: CT-1904

    • Fixed an issue where filtered events would trigger a useless update to the service trep_commit_seqno table while it is overwritten anyway once the last statement of the applied event is done, just prior to committing the whole block.

      Issues: CT-1931

    • Fixed an issue where the replicator would hang after applying a DROP TABLE event, that originally failed on the primary, but got logged into the binlog.

      Issues: CT-1973

    • Fixed a parsing issue that would prevent the replicator from correctly detecting a CREATE TABLE statement with START TRANSACTION

      Issues: CT-1987

  • Filters

    • Fixed an issue where the dropsqlmodes filter would fail to remove invalid sql modes from a multi-statement event

      Issues: CT-1993

  • Tungsten Connector

    • Connector now auto detects default authentication plugin by retrieving MySQL data source variable default_authentication_plugin rather than just using MySQL server version

      Issues: CT-1926

    • No longer printing warning "SequenceException: Parents differ" when canceling connections of a composite data service

      Issues: CT-1964

    • Fixed connector logging configuration to show hostname and class printing logs

      Issues: CT-1965

  • Tungsten Manager

    • The cctrl command datasource <ds> slave now sets the replicator role correctly. Previously, only the datasource role would change.

      Issues: CT-1882

    • A bug has been fixed that, in a few very rare cases, would allow replicas to continue to pull and apply THL from a failed primary whilst a failover was in the process of electing a new primary. This resulted in failovers being unable to complete fully. Whilst the new primary would be online and functioning, existing replicas in the cluster could experience errors due to THL discrepencies between the old and new primary nodes.

      Issues: CT-1986

  • API

    • Fixed REST API /api/v2/manager/control/service/{service}/switch call. It will not switch anymore to a shunned node.

      Issues: CT-796

    • Calls to /api/v2/manager/cluster/status now return properly when a peer cluster is fully offline or unreachable.

      Issues: CT-1945

    • The REST API call /api/v2/manager/control/service/{service}/datasource/{datasource}/slave now sets the role of the replicator correctly.

      Issues: CT-1975

1.5. Tungsten Clustering 7.0.1 GA (13 June 2022)

Version End of Life. Not Yet Set

Release 7.0.1 is the first minor release for the v7 release. This release contains a number of new features, minor bug fixes and a few key behavior changes, but importantly contains a critical bug fix for environments that use mixed-case table and database names, that could cause replication failures.

Customers already running v7.0.0 in production enviornments should upgrade to this release as soon as possible

Behavior Changes

The following changes have been made to Tungsten Cluster and may affect existing scripts and integration tools. Any scripts or environment which make use of these tools should check and update for the new configuration:

  • Installation and Deployment

    • A better error message is now provided when keystore password and enclosed private key password don't match.

      Issues: CT-1853

  • Command-line Tools

    • The cctrl output now displays the SSL status of the group communication and the connection to the database.

      $ cctrl 
      Tungsten Clustering 7.0.0 
      nyc: session established, encryption=true, authentication=true 
      jgroups: [un]encrypted, database: [un]encrypted

      Issues: CT-1785, CT-1832

    • The tpm ask summary command is now sorted by key when the Perl Tie::IxHash module is installed.

      Issues: CT-1820

    • The following commands which call cctrl now support the pass-through -t seconds timeout argument which specifies how long cctrl will wait for a connection to the Manager process before aborting:

      • check_tungsten_latency

      • check_tungsten_online

      • check_tungsten_policy

      • check_tungsten_services

      • tungsten_show_processlist

      • zabbix_tungsten_latency

      • zabbix_tungsten_online

      • zabbix_tungsten_progress

      • zabbix_tungsten_services

      Issues: CT-1822

    • The tpm diag command now gathers the output of the tpm ask summary command

      Issues: CT-1827

    • The tmonitor command now calls sudo internally as needed, removing the need for the tmonitor command to be run with sudo for the install and remove sub-commands.

      Issues: CT-1846

    • The tpm generate-haproxy-for-api default starting port is now 8091.

      Issues: CT-1865

    • The tungsten_find_orphaned command now places the log file in $CONTINUENT_ROOT/service_logs/, not /tmp/

      Issues: CT-1866

  • Core Replicator

    • The default value for the tpm property repl-svc-fail-on-zero-row-update has been changed from warn to stop

      Warning

      This is a significant change in behavior and could cause replicators, specifically cross-site replicators, in an Active/Active environment to stop and error more frequently.

      To maintain original behavior you will need to add this property with the value of warn to your configuration, providing you fully understand the inherent risk of ignoring such warnings.

      Issues: CT-1856

  • API

    • The manager REST API /api/v2/manager/status call now returns information regarding to security.

      Issues: CT-1830

Known Issue

The following issues are known within this release but not considered critical, nor impact the operation of Tungsten Cluster. They will be addressed in a subsequent patch release.

  • Installation and Deployment

    • After starting up Tungsten components, a defunct process for each running component can be found in the process listing.

      Whilst this does not cause any issues, it could generate unnecessary alerts for customers monitoring.

      The cause has been identified and affects version 7.0.0 and 7.0.1. This will be fixed in the next 7.0.2 release.

      Issues: CT-1876

Improvements, new features and functionality

  • Command-line Tools

    • Added the ability to turn auto recovery on or off dynamically, removing the need to run tpm update.

      This is done by running the following command:

      shell> trepctl -service servicename setdynamic -property replicator.autoRecoveryMaxAttempts -value <number>

      Note

      The service must be offline before changing the property

      Issues: CT-1088

    • The new command tungsten_upgrade_manager will help clear specific cached SSL values, to be used only when specified by Continuent Support.

      Issues: CT-1788

    • A new tpm report sub-command has been added. By default, tpm report will generate a security report for all available communication channels on a per-node basis. The purpose of tpm report is to provide easy access to all of the settings that pertain to a specific topic. The default (and only) topic at this time is the security stance. More topics will be added over time.

      Issues: CT-1793, CT-1857

    • The tapi command now supports the creation and modification of multiple APIv2 admin users. Two new cli args have been added: --create-user and --create-pass.

      Issues: CT-1801

    • The tapi command now auto-detects the protocol, either http or https

      Issues: CT-1803

    • Added connector<->manager link encryption status in the output of router * status command in cctrl

      Issues: CT-1811

    • Added JMX encryption and authentication status to trepctl -verbose and connector command outputs

      Issues: CT-1813

    • The tapi command will now output service names with the new option --includeServiceName, or if there is more than one service name found.

      Issues: CT-1860

  • Core Replicator

    • The thl list command now displays an approximative field size in bytes for row-based replication.

      Issues: CT-1847

  • Tungsten Connector

    • Added a connector mode command to print which mode the connector is running in, either "bridge" or "proxy"

      Issues: CT-1799

  • Security

    • Replaced official log4j library with a secured version in which all vulnerable classes have been removed. This prevents exposing the software following a (user) misconfiguration of the log4j properties

      Issues: CT-1810

  • API

    • Added a way to retrieve service properties through the API, for example, to check a specific service parameter, run the following API call :

      shell> curl https://127.0.0.1:8097/api/v2/replicator/service/alpha/properties?filter=replicator.global.db.sslEnabled
      { 
       "payloadType": "PropertiesListPayload", 
       "payloadVersion": "1", 
       "payload": { 
       "replicator.global.db.sslEnabled": "false" 
       } 
      }

      Issues: CT-1802

    • Added new top level REST API call to display RMI SSL status, run the following API call :

      shell> curl https://127.0.0.1:8097/api/v2/rmiSecurity

      Issues: CT-1812

Bug Fixes

  • Installation and Deployment

    • The tpm update command now properly handles v7 -> v7 upgrades.

      Issues: CT-1815

    • deployall script now properly displays the executable prefix for restarting services, typically mm_treplicator in Multi-Site/Active-Active replicator-only installs, as well as the full path to component binaries

      Issues: CT-1835

    • tpm would fail on keystores containing special characters in alias names.

      Issues: CT-1852

  • Command-line Tools

    • Worked around a ruby issue where DNS name resolution was sometimes preferred to file-based resolution in TPM. The following logic now applies: first trying file-based host name resolution, upon failure try DNS, then through ping command

      Issues: CT-1653

    • The tpm update command now returns clusters to the proper mode (i.e. maintenance or automatic) for staging-method upgrades (INI not affected).

      Issues: CT-1784

    • The tpm policy command now returns the proper policy in a composite cluster. There is also a new cli argument, --all, to show the policy for all cluster services.

      Issues: CT-1787

    • The tpm update command no longer aborts during a staging deployment when the actual hostname does not match what is configured for the node names in tpm

      Issues: CT-1791

    • Improves the tpm diag command so that it waits 2 seconds for replicator thread dump to complete.

      Issues: CT-1792

    • Fixed an `Undefined subroutine` call error in tapi --create

      Issues: CT-1798

    • The tpm connector command now properly passes the -e arguments to the mysql command.

      Issues: CT-1816

    • tpm now properly reports errors upon timeout executing commands. Typically, when a host is down, when upgrading or installing, tpm will now properly report that pinging this host failed.

      Issues: CT-1819

    • The tpm update command now properly determines services to be deleted.

      Issues: CT-1825

    • TPM would fail to validate data sources when using ssl-capath [directory] rather than ssl-ca [file] in my.cnf.

      This fixes only the validation part, automated truststore creation by TPM will not be possible at this point, a full tungsten-truststore.ts will have to be passed through --java-truststore-path

      Issues: CT-1826

    • The tapi command now auto-switches protocol (http or https) properly.

      Issues: CT-1829

    • replicator start offline would not properly pass the offline argument to systemd scripts when installed through deployall.

      Issues: CT-1836

    • undeployall script would not properly uninstall systemd services on some distributions including amazon linux.

      Issues: CT-1845

  • Backup and Restore

    • Fixed an issue with latest xtrabackup 8.0.28

      Issues: CT-1838

    • During the backup/restore process the MySQL certificates are kept intact.

      Issues: CT-1841

    • Restore will now succeed even if only one incremental backup has been created.

      Issues: CT-1842

    • Incremental backups can now be restored more then once.

      Issues: CT-1850

  • Core Replicator

    • Fixed the EnumToString and pkey filters to renew their database connections (every hour by default). This can be changed with the following properties:

      property=replicator.filter.enumtostring.reconnectTimeout=3600
      property=replicator.filter.pkey.reconnectTimeout=3600

      Issues: CT-1786

    • tpm update would fail with message "The host is configured to be '' but it is currently operating as a 'master'" when a dynamic replicator configuration file is found

      Issues: CT-1848

    • Fixes issues with PrimaryKeyFilter:

      • It could fail from correctly fetching primary key columns for tables with mixed upper and lower case letters in table names

      • Alter user statements could make it fail

      Issues: CT-1861

    • Fixed a NullPointerException that could happen if a replicator was put offline during startup. This had mostly no consequence except the exception being logged.

      Issues: CT-1863

  • Tungsten Connector

    • Fixed the output of connector cluster-status by adjusting log4 appender.

      Issues: CT-1778

    • An exception could occur while forcing shutdown of router gateways.

      Issues: CT-1779

  • Tungsten Manager

    • Fixed an issue where the command system summary issued in cctrl would hang the managers in the cluster.

      Issues: CT-1849

1.6. Tungsten Clustering 7.0.0 GA (29 Mar 2022)

Version End of Life. Not Yet Set

Release 7.0.0 is a major release introducing many new features including a fully documented API. There are a large number of bug fixes and improvements in all areas of the product, and a number of key behavior changes, most significantly being Security is now enabled by default.

Behavior Changes

The following changes have been made to Tungsten Cluster and may affect existing scripts and integration tools. Any scripts or environment which make use of these tools should check and update for the new configuration:

  • Installation and Deployment

    • An RPM update now issues --replace-release

      Issues: CT-1708

  • Command-line Tools

    • The tpm diag command now uses tar czf instead of the zip command to compress the gathered files. The zip command is no longer a pre-requisite for tpm diag.

      Issues: CT-1253

    • tungsten_set_position has been deprecated and no longer available in this release. dsctl should be used instead.

      tungsten_provision_slave has now been renamed to tprovision

      Issues: CT-1302

    • The tungsten_post_process script functionality has been merged into the tpm post-process command. The tungsten_post_process script remains as a shell wrapper for tpm post-process.

      Issues: CT-1314

    • The tungsten_find_orphaned script now creates a log file every time it runs which is stored in the configured temporary directory (/tmp/ by default, tpm query values temp_directory). This is to allow for easier troubleshooting and visibility during automatic execution.

      Issues: CT-1447

    • tpm now accepts chrony as a valid time synchronization software

      Issues: CT-1462

    • The tpm diag command now uses the ss (socket status) command in place of netstat on SUSE and other operating systems that have deprecated netstat.

      Issues: CT-1483

    • The tpm diag command now gathers the /etc/os-release file when located. Also now using the ip command on systems where ifconfig and/or route is deprecated.

      Issues: CT-1496

    • Changed output of thl purge command when no lower and upper bounds are given from 'Deleting events where' to 'Deleting all events'.

      Issues: CT-1738

  • Backup and Restore

    • --user and --password options have been added to the following scripts:

      If manager-rest-api-authentication=true (the default if not explicitly disabled) then these two new options must be supplied otherwise the scripts will fail with the following error: "ERROR >> Manager REST API authentication needed. Please specify the user name and password."

      The internal ruby api.rb module can now also handle the manager-rest-api-ssl=true (the default if not explicitly disabled) and will use https instead of http to access the REST API

      Issues: CT-1311

    • From v8 of xtrabackup, the --stream=tar option was removed, meaning that backups could fail if using the newer release of the Percona tools

      In this release, the backups will now create the backup and then manually compress it

      Warning

      This change will increase the required disk space for backups to allow the post-backup compression to complete.

      Issues: CT-1346

    • tungsten_provision_slave has now been renamed to tprovision.

      Issues: CT-1436

    • Additional messaging has been added to the output displayed when running tprovision.

      Issues: CT-1689

  • Core Replicator

    • Replicator will now check after extracting a mysql STOP_EVENT whether a new binlog file was created, actually handling this event as a ROTATE EVENT.

      This decreases the time that would be needed after STOP_EVENT to extract a new event..

      Issues: CT-1349

  • Tungsten Connector

    • Optimized transaction parsing by removing SQL transaction start recognition: in-transaction state is found in server status flags.

      Issues: CT-1698

  • Tungsten Manager

    • A new logfile, console.log is now generated in the $CONTINUENT_ROOT/tungsten/tungsten-manager/logs directory which contains all output displayed via cctrl. This file will provide Continuent Support with more valuable information when assisting to diagnose support cases.

      Issues: CT-1499

  • Other Issues

    • Log files for each component now have the same date and time stamp format.

      Issues: CT-1669

  • API

    • API calls triggering configuration changes are protected by a flag, i-am-sure=true, in order to avoid unwanted, potentially dramatic, configuration changes. This applies to:

      • configuration/module/servicesmap

      • reset

      • offline

      • online

      • onhold

      • addDataService

      • addDataSource

      Issues: CT-1317

Known Issue

The following issues are known within this release but not considered critical, nor impact the operation of Tungsten Cluster. They will be addressed in a subsequent patch release.

  • Installation and Deployment

    • After starting up Tungsten components, a defunct process for each running component can be found in the process listing.

      Whilst this does not cause any issues, it could generate unnecessary alerts for customers monitoring.

      The cause has been identified and affects version 7.0.0 and 7.0.1. This will be fixed in the next 7.0.2 release.

      Issues: CT-1876

  • Command-line Tools

    • The check_tungsten_online command returns a Replicator offline error on active witness hosts.

      Issues: CT-1783

    • The tpm policy command may return the incorrect policy value for composite clusters.

      As a result of this known issue, the tungsten_reset_manager command will also have issues on composite clusters because it calls tpm policy.

      Issues: CT-1787

Improvements, new features and functionality

  • Installation and Deployment

    • Two new tpm options have been included as part of the new API in this release.

      The options are used for setting the API admin user credentials and are as follows:

      Issues: CT-1327

    • Support now included for MariaDB 10.3+

      Issues: CT-1276, CT-1433

    • Support has been added for Java 17 LTS

      Issues: CT-1706

  • Command-line Tools

    • A new tpm option delete-service is now available to simplify the removal of clusters and/or replicator services.

      Issues: CT-210, CT-327, CT-1275

    • Prometheus exporters mysqld_exporter and node_exporter are now included with the distribution packages.

      A new command line tool tmonitor is now available for the management and testing of external Prometheus exporters (node and mysqld), and for the testing of internal exporters (Manager, Connector and Replicator).

      Issues: CT-960

    • A new tpm option purge-thl and a new script tungsten_purge_thl have been added to allow easier and more intelligent THL purging across all nodes in a topology.

      This allow you to purge THL files based on the following rules:

      • Gather the last applied seqno from all Replica nodes and take the lowest one

      • Find the current THL file which contains that seqno, then locate the previous one

      • Construct a thl purge command to remove thl thru the last seqno in the prev file

      The default behavior is to display the needed commands for the admin to execute manually.

      Issues: CT-1273

    • tpm diag now collects routing table information via route -n , and has two new command-line arguments: --include and --groups.

      --include specifies a comma-separated list of subroutines to include. Any gather subroutine not listed will be skipped.

      --groups specifies a comma-separated list of subroutine groups to include. Any group not listed will be skipped.

      Issues: CT-1322

    • A new sub-command has been added, tpm generate-haproxy-for-api. This read-only action will read all available INI files and dump out corresponding haproxy.cfg entries with properly incrementing ports; the composite parent will come first, followed by the composite children in alphabetical order.

      The tungsten_generate_haproxy_for_api script functionality has been merged into the tpm generate-haproxy-for-api command. The tungsten_generate_haproxy_for_api script remains as a shell wrapper for tpm generate-haproxy-for-api.

      Issues: CT-1342

    • tungsten_send_diag now supports a new command-line argument, --cleanup, which will cause the removal of the diagnostic archive file generated using the --diag argument.

      Issues: CT-1360

    • The tungsten_reset_manager command is now able to restart the Manager process when the --start or -s argument is passed in.

      Issues: CT-1401

    • With the release of APIv2, a new cli tool has been introduced to allow easier access called tapi.

      In addition, the vast majority of Tungsten cli tools have been updated to optionally use the APIv2 interface when desired.

      The Nagios and Zabbix checks are also available via APIv2 using the tapi tool.

      Issues: CT-1454

    • The tungsten_purge_thl command is now a wrapper for the tpm purge-thl command.

      Issues: CT-1488

    • The tmonitor command now has better help text and more options to ease usage, including --filter to allow easy viewing of the tmonitor test output.

      Issues: CT-1585

    • A new option to print the merged logs to STDOUT has been added to tungsten_merge_logs (--stdout|-O).

      The tpm command suite now properly supports the --profile argument to specify a Tungsten json configuration file in place of the installed tungsten.cfg.

      Issues: CT-1680

    • The tapi command now supports the --affinity argument which will display all Connector-specific affinity settings, along with --connectorstatus to show all.

      Issues: CT-1700

    • The cctrl.log file is now accessible from the $CONTINUENT_ROOT/service_logs directory

      Issues: CT-1727

    • A new command (error) has been added to trepctl to output a full stack trace of the last error, if any.

      shell> trepctl -service <serviceName> error
      
      Event application failed: seqno=10 fragno=0 message=Table hr.regions not found in database. Unable to generate a valid statement.
      com.continuent.tungsten.replicator.applier.ApplierException: Table hr.regions not found in database. Unable to generate a valid statement.
      at com.continuent.tungsten.replicator.applier.JdbcApplier.getTableMetadata(JdbcApplier.java:582)
      at com.continuent.tungsten.replicator.applier.JdbcApplier.fillColumnNames(JdbcApplier.java:494)
      at com.continuent.tungsten.replicator.applier.JdbcApplier.getColumnInformation(JdbcApplier.java:1236)
      at com.continuent.tungsten.replicator.applier.MySQLApplier.applyOneRowChangePrepared(MySQLApplier.java:418)
      at com.continuent.tungsten.replicator.applier.JdbcApplier.applyRowChangeData(JdbcApplier.java:1460)
      at com.continuent.tungsten.replicator.applier.JdbcApplier.apply(JdbcApplier.java:1576)
      at com.continuent.tungsten.replicator.applier.ApplierWrapper.apply(ApplierWrapper.java:100)
      at com.continuent.tungsten.replicator.pipeline.SingleThreadStageTask.apply(SingleThreadStageTask.java:871)
      at com.continuent.tungsten.replicator.pipeline.SingleThreadStageTask.runTask(SingleThreadStageTask.java:601)
      at com.continuent.tungsten.replicator.pipeline.SingleThreadStageTask.run(SingleThreadStageTask.java:185)
      at java.base/java.lang.Thread.run(Thread.java:834)

      Issues: CT-1747

  • Backup and Restore

    • Added support for mariabackup. Configurable tpm option:

      backup-method=mariabackup
      or
      backup-method=mariabackup-incremental

      Issues: CT-1100

    • Fixes an issue where tprovision could remove the SSL certs for MySQL on reprovision

      Issues: CT-1323

  • Core Replicator

    • It is now possible to compress and/or encrypt THL on disk. For more information on using these features see https://docs.continuent.com/tungsten-clustering-7.0/thl-compress-encrypt.html

      Issues: CT-630

    • The replicator will now be able to handle new SQL_MODES available in later releases of MySQL and MariaDB, these are as follows:

      • MySQL: TIME_TRUNCATE_FRACTIONAL

      • MariaDB: TIME_ROUND_FRACTIONAL, SIMULTANEOUS_ASSIGNMENT

      Issues: CT-1362

    • In-Flight THL Compression is now available.

      For full details on enabling this feature, refer to this page

      Issues: CT-1420

  • Tungsten Connector

    • Audit Logging is now available in the Connector: it allows for logging data transferred between client application and MySQL servers to a file, database or socket.

      For more details and steps to enable, see https://docs.continuent.com/tungsten-clustering-7.0/connector-advanced-audit-logging.html

      Issues: CT-78

    • Added support for TLSv1.3.

      Note that TLS 1.3 was introduced in jdk 8u272. So tpm will only set it by default for java 8u272 and later versions. It is still possible to force the protocol to be used via tpm flag --tls-enabled-protocols=TLSv1.xxx

      Issues: CT-1367

    • A new Dynamic Active/Active feature has been added within the Proxy.

      This new Proxy mode is for Composite Active/Active clusters only, and allows you to specifically configure writes to be directed to a single cluster by the use of the tpm connector-write-affinity flag.

      For full details on this feature, and configuring it, see [link to be added]

      Issues: CT-1540

    • connector drain [opt-timeout] has been introduced as an alias to existing connector graceful-stop command and will prevent new connections, then shutdown the connector after an optional delay.

      The opt-timeout parameter (seconds) can be specified to limit the wait before stopping the connector. Not passing this parameter implies infinite wait.

      Issues: CT-1644

    • connector-reset-when-affinity-back is now available with proxy mode.

      Issues: CT-1763

  • Security

    • Tungsten can now install on default CentOS and RedHat 8 with tightened security settings thanks to support of TLSv1.3

      Issues: CT-1359

    • Tungsten software now undergoes a rigorous security scan during QA. We also check included open-source/3rd-party software.

      Issues: CT-1579

  • Monitoring

    • A number of new metrics have been added to the Prometheus exporters.

      Issues: CT-1266, CT-1615

  • Platform Specific Deployments

    • ARM 64 bit processor support added (linux aarch64)

      Note

      Note that at time of release, there is currently no xtrabackup binary available for ARM.

      Issues: CT-1619, CT-1620

  • Other Issues

    • General improvements in MySQL 8.0 support.

      Issues: CT-1346

    • IPv6 host addresses are now fully supported.

      Can be enabled with the following configuration property:

      prefer-ip-stack="6"

      By default, IPv4 is enabled, which equates to the value of "4" in the above property.

      Issues: CT-1537

Bug Fixes

  • Installation and Deployment

    • Default systemd configuration files for Tungsten components no longer specifies the tungsten group to execute the command. This will prevent file access issues when tungsten user belongs to several groups.

      Issues: CT-1550

    • When services are deployed with systemd and MySQL could not start due to an error, tpm would not be able to later start MySQL

      Issues: CT-1734

  • Command-line Tools

    • In certain cases, tprovision would not be able to find the binary log position of the backup when taken from a primary. This has been fixed.

      Issues: CT-1085

    • Fixes a bug in tprovision when using xtrabackup version 8, due to changes in xtrabackup binaries.

      Issues: CT-1248

    • The tpm connector command now handles special characters in the password string.

      Issues: CT-1258

    • The tpm update command will now exit with an error if any files not owned by the configured Tungsten OS user are found in the Tungsten installation directory.

      For example, if the OS user is tungsten and the installation directory is /opt/continuent, containing the file /opt/continuent/thl/archived_thl.zip owned by root would cause something like the following error to be produced, and tpm update would exit:

      Foreign-owned files found!
      
      Located files in the Tungsten installed directory /opt/continuent
      that are not owned by the Tungsten OS user (tungsten):
      
      /opt/continuent/thl/archived_thl.zip
      
      Please change the ownership of these files to OS user "tungsten"
      using the chown command as root via sudo, then rerun the `tpm update` command.
      
      For example:
      shell> sudo chown -R tungsten:tungsten /opt/continuent

      Issues: CT-1260

    • The tpm diag command now behaves correctly on Connector-only nodes, where previously it would try to gather Manager and Replicator-specific items.

      Issues: CT-1284

    • Fixes a security issue within the tpm diag command.

      Issues: CT-1295

    • The number of created and active connections could be incorrect when listed from a composite data service

      Issues: CT-1312

    • tungsten_send_diag no longer prints an error about Use of uninitialized value $diagArgs in concatenation.

      Issues: CT-1354

    • The tpm command no longer prints an error when run with no other command-line arguments.

      Issues: CT-1373

    • The tpm command no longer aborts with a Use of uninitialized value error when a stray tungsten.cfg file exists under $CONTINUENT_ROOT

      Issues: CT-1394

    • Fixes a monitoring bug with users using caching_sha2_password.

      Issues: CT-1406

    • tprovision (formerly tungsten_provision_slave) may fail to provision if the MySQL data directory was not accessible to the tungsten user.

      Issues: CT-1475

    • The tpm generate-haproxy-for-api command no longer fails on CentOS 8.

      Issues: CT-1484

    • The tmonitor command no longer fails on Debian 9 and Ubuntu.

      Issues: CT-1485

    • All tpm sub-commands now handle command-line arguments more intelligently.

      Issues: CT-1487

    • The tpm purge-thl command now handles command-line arguments more intelligently.

      Issues: CT-1489

    • The tpm diag command now properly collects the system information file on Debian systems.

      Issues: CT-1492

    • Database monitoring logs are now reporting the correct error number and SQL state when database errors occur.

      Issues: CT-1497

    • The tpm update command now handles updates/upgrades more gracefully when the previous version did not have the latest tpm framework.

      Issues: CT-1506

    • The tpm update command will now properly remove composite services.

      Issues: CT-1519

    • The tungsten_find_orphaned command no longer fails with an 'Can't exec "/bin/sh": Argument list too long' error when there are too many THL files to parse.

      Issues: CT-1545

    • The tpm ask command no longer calls Data::Dumper when it is not available.

      Issues: CT-1626

    • tpm now parses the MYSQL SSL related setting correctly.

      Issues: CT-1662

    • Fixes an issue where the deployall command would create a root owned wrapper.log in the ./tools directory.

      Issues: CT-1664

    • When MySql services where badly installed, some distribution could show a “not-found” status within systemctl, confusing tpm

      Issues: CT-1677

    • The tpm command now communicates properly when there is no INI configuration file or staging-method deploy.cfg configuration defined.

      Issues: CT-1712

    • The tpm diag command now handles Multi-Site/Active-Active topologies better.

      Issues: CT-1718

    • tungsten_monitor.rb script no longer uses sudo to send emails if the configuration doesn't allow it.

      Issues: CT-1737

    • The tpm diag command now handles zero-length mysqld.log files gracefully.

      Issues: CT-1740

  • Backup and Restore

    • When running the cluster_backup in a Active/Active environment, when setting require_master_backup to false, the script would still attempt to backup the Primary as it would scan the wrong sub-service and incorrectly identify the Relay node as a candidate.

      Issues: CT-1280

    • Fixed an issue where an xtrabackup generated by the replicator would fail to be restored using trepctl restore command.

      Issues: CT-1575

  • Core Replicator

    • The replicator metadata cache will now correctly handle table names when lower_case_table_names=1 is set in the MySQL configuration.

      Issues: CT-651

    • When using parallel apply, the replicator would error with a Foreign Key constraint error if statements were issued against two or more objects that shared the same name, but with different case sensitivity, for example:

      mysql> create table testtable;
      mysql> drop table testtable;
      mysql> create table TestTable;

      Issues: CT-1259

    • A change in the way MySQL logs CREATE TABLE AS SELECT in the Binary Logs from v8.0.20 onwards, meant these transactions would previously fail.

      Warning

      Whilst these statements will now replicate, it mut be noted that in the event of a failure during the data load, the initial CREATE statement won't be rolled back, and therefore care must be taken when using this type of DDL.

      Note

      This only affects customers using MySQL v8.0.20+ running with ROW based replication. An alternative workaround to ensure correct rollback on failure, would be to run the statement with STATEMENT based replication for the session. This will also provide better performance for larger tables.

      Issues: CT-1301

    • Fixes occurences of NullPointerException that would occur when bringing the replicator online before MySQL was started.

      Issues: CT-1348

    • For row based events, SQL modes were not displayed in the THL output. This is now fixed.

      Issues: CT-1440

    • When connecting to a THL server, a client will now connect to the next available host in its THL uri, if the first does not have the sequence number that the client requires. The client will then fail only if none of the hosts from the uri can provide the needed sequence number.

      Issues: CT-1558

    • Fixed an issue when using Parallel apply that would show a NullPointerException in case an event could either not be found or be corrupted in THL. This will now display a correct message Missing or corrupted event from storage

      Issues: CT-1722

    • Fixed an issue where trepctl was leaving JMX connections opened.

      Issues: CT-1752

    • Added more debug information for detecting possible hanging connections while a THL client connects to the THL server. Also, added socket timeout for the connection initialization

      Issues: CT-1760

  • Filters

    • Includes previously missing template file to enable easy configuration of the dbrename filter.

      Issues: CT-1350

    • The BidiRemoteSlaveFilter could fail to correctly flag fragmented events in unprivileged environments (Aurora, for example) In such an environment (multi-active, unprivileged database access), a new setting was introduced to force extraction process to read ahead to the last fragment to detect the service name (false by default). Enabled with repl_svc_extractor_multi_frag_service_detection=true

      Issues: CT-1351

  • Tungsten Connector

    • By default, the connector will no longer transparently reconnect underlying connections to database servers when the data service changes.

      This will prevent the following case: in Composite Active/Active topologies, a given connection starts to write data to a site. The site fails, connection gets reconnected the other site and resumes writing. However the data written to the 1st site has not reached the 2nd site, thus data will not be consistent.

      Default is to reject reconnections that follow a write operation (RW_STRICT connection or SmartScale after a write) and to allow reconnection after a read operation (RO_RELAXED or SmartScale after a read) which translates to --connector-allow-cross-site-reconnects-for-writes=false and --connector-allow-cross-site-reconnects-for-reads=true

      It is still possible to get the previous behavior (reconnecting transparently connections cross-site) by specifying both --connector-allow-cross-site-reconnects-for-writes=true and --connector-allow-cross-site-reconnects-for-reads=true, at your own risk

      Issues: CT-1265

    • With @direct r/w splitting in active/active configurations, the connector was not correctly redirecting connections to an active site after re-joining the cluster following a failure.

      Issues: CT-1400

    • The Connector now properly retries specific MySQL commands when possible: INIT_DB, CHANGE_USER, STATISTICS and prepared statement functions.

      Issues: CT-1480

    • Connector now forbids non-ssl connections when mysql server has require_secure_transport=ON

      Issues: CT-1666

    • Connector now mirrors the MySQL default connect_timeout by retrieving it from the primary when starting up. This timeout will apply to all connections made from the connector to MySQL servers.

      This setting can be over-ridden by using the following tpm property

      property=connectTimeout=VALUE

      If VALUE set to autodetect, this value will mirror the MySQL connect_timeout system variable. Set to 0 for infinite timeout.

      Issues: CT-1726

    • Connector no longer requests cross-site services information from manager. In addition to removing extra network traffic, this cures a problem of connections counted twice in data source connection statistics

      Issues: CT-1775

  • Tungsten Manager

    • Fixes a NullPointerException error (NPE) in the manager logs.

      Issues: CT-1132

    • Fixes an edge case bug that would allow a Composite Active/Active cluster to contain 2 Primary nodes.

      Issues: CT-1474

    • Fixes a manager-internal connection check during recovery that was not properly using SSL when required.

      Issues: CT-1661

    • Early initialisation of the REST API sometimes caused the manager to hang and fail to startup correctly on new installations.

      Issues: CT-1725

    • Fixes an issue when the manager would wait longer than the timeout for replicator purge on a failover.

      Issues: CT-1733

    • Fixes an issue where datasource <datasourcename> welcome would fail to welcome a manually failed composite datasource.

      Issues: CT-1771

  • Security

    • log4j libraries updated to v2.17.1 specifically to mitigate risk of exposure to the 0-day vulnerbaility detected in log4j v2.14

      Issues: CT-1703

1.7. Tungsten Clustering 6.1.24 GA (11 Dec 2023)

Version End of Life. 15 Aug 2024

Release 6.1.24 contains a critical bug fix to address an issue during failover.

Bug Fixes

  • Tungsten Manager

    • Fixes an issue when the manager would wait longer than the timeout for replicator purge on a failover.

      Issues: CT-1733

1.8. Tungsten Clustering 6.1.23 GA (31 Aug 2023)

Version End of Life. 15 Aug 2024

Release 6.1.23 contains a number of critical bug fixes.

Bug Fixes

  • Tungsten Connector

    • Fixed an issue where a failover could hang in rare cases when security and Proxy mode are enabled. SSL connections to a failed data source could enter a deadlock and block failover for up to 15 minutes.

      Note

      This issue only affects users running Java 11, and is related to https://bugs.openjdk.org/browse/JDK-8241239

      Issues: CT-2183, CT-2187

1.9. Tungsten Clustering 6.1.22 GA (6 July 2023)

Version End of Life. 15 Aug 2024

Release 6.1.22 contains a number of critical bug fixes and it is advised customers running 6.1.21 should upgrade to this release.

Behavior Changes

The following changes have been made to Tungsten Cluster and may affect existing scripts and integration tools. Any scripts or environment which make use of these tools should check and update for the new configuration:

  • Command-line Tools

    • cctrl now accepts services names with capital letters, dots and hyphens

      Issues: CT-2163

Bug Fixes

  • Installation and Deployment

    • Fixed RPM package script to run tpm install instead of tpm update when installing the rpm

      Issues: CT-2130

  • Core Replicator

    • A change in the way MySQL logs CREATE TABLE AS SELECT in the Binary Logs from v8.0.20 onwards, meant these transactions would previously fail.

      Warning

      Whilst these statements will now replicate, it mut be noted that in the event of a failure during the data load, the initial CREATE statement won't be rolled back, and therefore care must be taken when using this type of DDL.

      Note

      This only affects customers using MySQL v8.0.20+ running with ROW based replication. An alternative workaround to ensure correct rollback on failure, would be to run the statement with STATEMENT based replication for the session. This will also provide better performance for larger tables.

      Issues: CT-1301

    • Fixed an issue where an extracted JSON field could be invalid, with doubled commas in JSON arrays.

      Issues: CT-2049

    • Improved a query that is run by Tungsten when fetching tables metadata (column names, datatypes, etc). While it is not generally needed, the unoptimized query can run badly (especially) against old mysql versions with a lot of databases / tables. For now, the new optimized query is not used by default, but this could change in some future version.

      This can be enabled by using the following property :

      property=replicator.datasource.global.connectionSpec.usingOptimizedMetadataQuery=true

      Issues: CT-2077

  • Tungsten Connector

    • Note, this bug only affects v6.1.21

      Removed extra logging for each "SET xxx" statement when in proxy mode. On top of polluting the logs, this extra logging can overwhelm the connector, making it slow to respond and have client threads pile up, up to the point where that could exhaust memory

      Issues: CT-2165

1.10. Tungsten Clustering 6.1.21 GA (18 Apr 2023)

Version End of Life. 15 Aug 2024

Release 6.1.21 contains a number of critical bug fixes and improvements.

Behavior Changes

The following changes have been made to Tungsten Cluster and may affect existing scripts and integration tools. Any scripts or environment which make use of these tools should check and update for the new configuration:

  • Command-line Tools

    • The tungsten_skip_all command (along with alias tungsten_skip_seqno) now shows the full pendingExceptionMessage instead of just pendingError, and the More choice shows the pendingErrorEventId and the pendingError.

      Issues: CT-2030

    • Enabling a custom backup script does not require it to be made the default option anymore, i.e. Setting --repl-backup-method to the custom backup script is not required anymore.

      Issues: CT-2039

    • No longer using connector graceful-stop together with systemd for upgrades. The underlying change of binary confuses systemd scripts.

      Issues: CT-2113

Known Issue

The following issues are known within this release but not considered critical, nor impact the operation of Tungsten Cluster. They will be addressed in a subsequent patch release.

  • Tungsten Connector

    • Note, this bug only affects this release, v6.1.21

      Each "SET xxx" statement issued when in proxy mode is being logged. On top of polluting the logs, this extra logging can overwhelm the connector, making it slow to respond and have client threads pile up, up to the point where that could exhaust memory

      If using Proxy mode, it is advised to NOT install this release, and instead use either version 6.1.20 or 6.1.22

      Issues: CT-2165(ki)

Improvements, new features and functionality

  • Command-line Tools

    • tungsten show processlist could error with a disconnection message listing connections disconnected on the mysql server side.

      Issues: CT-2112

    • The tpm diag command now captures the output of cctrl> cluster topology validate

      Issues: CT-2115

  • Platform Specific Deployments

    • ARM 64 bit processor support added (linux aarch64)

      Note

      Note that at time of release, there is currently no xtrabackup binary available for ARM.

      Issues: CT-1619, CT-1620

Bug Fixes

  • Command-line Tools

    • The tungsten_get_status command now handles shunned nodes properly.

      Issues: CT-2026

    • The tpm ask keys command now lists all available keys properly.

      Issues: CT-2041

1.11. Tungsten Clustering 6.1.20 GA (19 Dec 2022)

Version End of Life. 15 Aug 2024

Release 6.1.20 contains a number of critical bug fixes and improvements.

Behavior Changes

The following changes have been made to Tungsten Cluster and may affect existing scripts and integration tools. Any scripts or environment which make use of these tools should check and update for the new configuration:

  • Command-line Tools

    • Both the tungsten_get_ports and tpm report commands have been updated to use the ss OS command when the netstat OS command is unavailable or deprecated.

      Issues: CT-2007

  • Core Clustering

    • Changed the behavior of the connector and manager upon local site failover:

      No longer sending write traffic to the remote site unless the local site is fully offline. In case of local failover, the connector will now pause connections until a new primary is elected. This will avoid risks of out-of-order apply after local failover

      Issues: CT-2009

Improvements, new features and functionality

  • Command-line Tools

    • A new command has been added called tungsten_get_ports, which will display the running Tungsten processes and the associated TCP ports that those processes are listening on.

      Issues: CT-1970

    • The tungsten_get_ports command now supports the /opt/continuent/tungsten/tungsten-connector/conf/interfaces.json file, if found.

      Issues: CT-2003

    • The tungsten_get_ports command now use values based on values obtained from tpm query values when possible for the greatest output accuracy.

      Issues: CT-2008

Bug Fixes

  • Command-line Tools

    • Worked around a ruby issue where DNS name resolution was sometimes preferred to file-based resolution in TPM. The following logic now applies: first trying file-based host name resolution, upon failure try DNS, then through ping command

      Issues: CT-1653

  • Backup and Restore

    • When running the cluster_backup in a Active/Active environment, when setting require_master_backup to false, the script would still attempt to backup the Primary as it would scan the wrong sub-service and incorrectly identify the Relay node as a candidate.

      Issues: CT-1280

  • Filters

    • Fixed an issue where the dropsqlmodes filter would fail to remove invalid sql modes from a multi-statement event

      Issues: CT-1993

  • Tungsten Manager

    • The cctrl command datasource <ds> slave now sets the replicator role correctly. Previously, only the datasource role would change.

      Issues: CT-1882

    • A bug has been fixed that, in a few very rare cases, would allow replicas to continue to pull and apply THL from a failed primary whilst a failover was in the process of electing a new primary. This resulted in failovers being unable to complete fully. Whilst the new primary would be online and functioning, existing replicas in the cluster could experience errors due to THL discrepencies between the old and new primary nodes.

      Issues: CT-1986

1.12. Tungsten Clustering 6.1.19 GA (17 Oct 2022)

Version End of Life. 15 Aug 2024

Release 6.1.19 contains a number of critical bug fixes and improvements, specifically for customers using Parallel Replication.

Behavior Changes

The following changes have been made to Tungsten Cluster and may affect existing scripts and integration tools. Any scripts or environment which make use of these tools should check and update for the new configuration:

  • Command-line Tools

    • The check_tungsten.sh was deprecated in release 6.1.18 and has now been removed from this release.

      Issues: CT-1939

  • Core Replicator

    • repl_svc_extractor_multi_frag_service_detection is now turned ON by default. Event shards are determined at extraction time. With fragmented events, the shard cannot be determined by only reading the first fragment, but needs to check the last fragment as well. With this setting turned OFF, there is no issue with pipelines that don't need it, i.e. no parallel apply downstream replicas. However, as this is done at extract time, THL contains this information, and adding or changing a replica using parallel apply could introduce issues.

      Note

      It can be disabled if you see a performance overhead but this should be done with caution. For Aurora<>Aurora Active/Active deployments it is essential that this property be left ON.

      Issues: CT-1959

  • Tungsten Connector

    • Improved tungsten show processlist by running underlying commands in parallel.

      Issues: CT-1569

Improvements, new features and functionality

  • Command-line Tools

    • The tpm command calls to glob have been improved to be more strict and compliant.

      Issues: CT-1940

    • The tpm ask stages and tpm ask allstages commands have been added to display the Replicator stages for the current node (stages) and the stages for each role (allstages).

      Issues: CT-1943

    • The tpm ask command has five new variables available: dsrole & dsstate for the current datasource, and trrole & trstate for the current replicator, and nodeinfo which displays all 4 of the new variables.

      Issues: CT-1944

    • A new standalone status script has been added called tungsten_get_status that shows the datasources and replicators for all nodes in all services along with seqno and latency.

      Issues: CT-1962

  • Tungsten Connector

    • Added a connector mode command to print which mode the connector is running in, either "bridge" or "proxy"

      Issues: CT-1799

    • The connector graceful-stop command now supports systemd service manager properly. The connector stop command now takes an optional argument that will make it a graceful stop. If connector stop is run without the parameter, it will stop the connector immediately. If a positive number of seconds is passed, it will wait, at most, this timeout for connections to disconnect (refusing new connections), after which it will force close all connections and shutdown the connector. connector graceful-stop behavior is unchanged: without the parameter, the connector will wait "forever" for connections to disconnect. A positive timeout in seconds can be passed to sever connections after the given delay

      Issues: CT-1921

Bug Fixes

  • Installation and Deployment

    • The tpm install and tpm update commands now properly support the --thl-ports option for cross-site subservices.

      Issues: CT-1953

  • Command-line Tools

    • Fixes an issue where the pause state would be badly displayed under no load, when the pause reaches the defined time. Note this is only a display issue : once an event is received, the pause state and displayed remaining time will be reseted correctly.

      Issues: CT-1942

    • The tpm mysql command will now gracefully handle being run on a non-database node.

      Issues: CT-1946

  • Core Replicator

    • Fixes an issue that would leave a transaction uncommitted longer than necessary.

      Issues: CT-1958

    • Fixed an issue where the shard_id stored in the trep_commit_seqno table would show an incorrect shard after processing a skipped event (an event from another channel)

      Issues: CT-1967

    • Fixed an issue where the replicator would hang after applying a DROP TABLE event, that originally failed on the primary, but got logged into the binlog.

      Issues: CT-1973

  • Filters

    • The BidiRemoteSlaveFilter could fail to correctly flag fragmented events in unprivileged environments (Aurora, for example) In such an environment (multi-active, unprivileged database access), a new setting was introduced to force extraction process to read ahead to the last fragment to detect the service name (false by default). Enabled with repl_svc_extractor_multi_frag_service_detection=true

      Issues: CT-1351

  • Tungsten Connector

    • Connector now auto detects default authentication plugin by retrieving MySQL data source variable default_authentication_plugin rather than just using MySQL server version

      Issues: CT-1926

1.13. Tungsten Clustering 6.1.18 GA (7 Sept 2022)

Version End of Life. 15 Aug 2024

Release 6.1.18 contains a number of minor bug fixes and improvements.

Behavior Changes

The following changes have been made to Tungsten Cluster and may affect existing scripts and integration tools. Any scripts or environment which make use of these tools should check and update for the new configuration:

  • Command-line Tools

    • The check_tungsten.sh script has been deprecated and no longer maintained. It will be removed in the next patch release and users should use the newer tmonitor commands.

      Issues: CT-1038

    • The tungsten_find_orphaned command now places the log file in $CONTINUENT_ROOT/service_logs/, not /tmp/

      Issues: CT-1866

  • Backup and Restore

    • From v8 of xtrabackup, the --stream=tar option was removed, meaning that backups could fail if using the newer release of the Percona tools

      In this release, the backups will now create the backup and then manually compress it

      Warning

      This change will increase the required disk space for backups to allow the post-backup compression to complete.

      Issues: CT-1346

  • Tungsten Manager

    • A failsafe shunned cluster (Caused by a network split) will be auto recovered after the network connection is re-established.

      Issues: CT-241

Improvements, new features and functionality

  • Command-line Tools

    • The tungsten_merge_logs command now supports the --before TIMESTAMP and --after TIMESTAMP filters

      Issues: CT-1869

    • The tungsten_generate_haproxy_for_api and tpm generate-haproxy-for-api commands now support using connector hosts in the backend definitions via -c, and extra backend flags to the backend hosts lines using -f.

      Issues: CT-1909

    • The tungsten_generate_haproxy_for_api and tpm generate-haproxy-for-api commands no longer call the Perl Data::Dumper module.

      Issues: CT-1915

    • The tpm ask isBridge and tpm ask isCoordinator commands have been back-ported to v6.

      Issues: CT-1916

    • The tungsten_reset_manager command now supports the ability to simply print out the path or paths to be cleared, one per line via the -l or --list arguments.

      Issues: CT-1917

  • Core Replicator

    • The thl list command now displays an approximative field size in bytes for row-based replication.

      Issues: CT-1847

    • Added a new feature that enables pausing a replicator stage for some amount of time.

      This will pause the given stage for 100 seconds.:

      trepctl pause -stage thl-to-q -time 100

      This will pause the stage indefinitely (or until restart, etc) Add -y to avoid the prompt message whether you are sure.

      trepctl pause -stage thl-to-q

      For the previous 2 commands, running a pause command again will override the previous command.

      This will resume the suspended stage (Note that if the stage is not paused, this will have no effect):

      trepctl resume -stage thl-to-q

      Note

      Please note this pause does not survive a replicator restart or a service offline/online.

      Issues: CT-1912

  • Tungsten Manager

    • Added a new option to TPM manager-replicator-offline-timeout=<timeout_in_sec> that configures the timeout for the manager to wait until the replicator goes offline. When parallel applier is in use the default timeout was too low, so it’s now user configurable so that it can be adjusted to suit different topologies. If not supplied, the default is 180 (3 minutes). This value should be sufficent in most use cases.

      Issues: CT-1892

Bug Fixes

  • Installation and Deployment

    • Fixes issues where fixed properties and filters passed to tpm in service stanzas were not being configured correctly

      Issues: CT-1463

  • Command-line Tools

    • replicator start offline would not properly pass the offline argument to systemd scripts when installed through deployall.

      Issues: CT-1836

    • undeployall script would not properly uninstall systemd services on some distributions including amazon linux.

      Issues: CT-1845

    • The tpm diag command now passes when the nodename defined in the tungsten.ini is the shortname, and DNS returns the FQDN.

      Issues: CT-1908

    • The tpm diag command now gathers the mysql.log file when SSL is enabled in the server

      Issues: CT-1933

    • The tpm diag command now behaves properly when the MySQL server is logging to STDERR.

      Issues: CT-1935

  • Backup and Restore

    • Fixed an issue where an xtrabackup generated by the replicator would fail to be restored using trepctl restore command.

      Issues: CT-1575

    • During the backup/restore process the MySQL certificates are kept intact.

      Issues: CT-1841

    • Restore will now succeed even if only one incremental backup has been created.

      Issues: CT-1842

    • Incremental backups can now be restored more then once.

      Issues: CT-1850

  • Core Replicator

    • Fixes an issue that would prevent a service from going offline at a specified time (trepctl online -until-time) when parallel apply is enabled. This is a rework of CT-1243.

      Issues: CT-1684

    • Fixed a possible issue when recovering an old primary as a replica after failover when parallel apply is enabled, that could lead the replica to be unable to come online and require a reprovisioning of this replica.

      Issues: CT-1890

    • Fixes an issue that prevented geometry datatypes with SRID from being replicated.

      Issues: CT-1904

    • Fixed an issue where filtered events would trigger a useless update to the service trep_commit_seqno table while it is overwritten anyway once the last statement of the applied event is done, just prior to committing the whole block.

      Issues: CT-1931

1.14. Tungsten Clustering 6.1.17 GA (16 May 2022)

Version End of Life. 15 Aug 2024

Release 6.1.17 contains a number of minor bug fixes and improvements.

Behavior Changes

The following changes have been made to Tungsten Cluster and may affect existing scripts and integration tools. Any scripts or environment which make use of these tools should check and update for the new configuration:

  • Command-line Tools

    • Changed output of thl purge command when no lower and upper bounds are given from 'Deleting events where' to 'Deleting all events'.

      Issues: CT-1738

    • The following commands which call cctrl now support the pass-through -t seconds timeout argument which specifies how long cctrl will wait for a connection to the Manager process before aborting:

      • check_tungsten_latency

      • check_tungsten_online

      • check_tungsten_policy

      • check_tungsten_services

      • tungsten_show_processlist

      • zabbix_tungsten_latency

      • zabbix_tungsten_online

      • zabbix_tungsten_progress

      • zabbix_tungsten_services

      Issues: CT-1822

    • The tpm diag command now gathers the output of the tpm ask summary command

      Issues: CT-1827

  • Backup and Restore

    • Additional messaging has been added to the output displayed when running tprovision.

      Issues: CT-1689

Improvements, new features and functionality

  • Command-line Tools

    • Added the ability to turn auto recovery on or off dynamically, removing the need to run tpm update.

      This is done by running the following command:

      shell> trepctl -service servicename setdynamic -property replicator.autoRecoveryMaxAttempts -value <number>

      Note

      The service must be offline before changing the property

      Issues: CT-1088

    • The cctrl.log file is now accessible from the $CONTINUENT_ROOT/service_logs directory

      Issues: CT-1727

  • Tungsten Connector

    • connector-reset-when-affinity-back is now available with proxy mode.

      Issues: CT-1763

  • Security

    • Replaced official log4j library with a secured version in which all vulnerable classes have been removed. This prevents exposing the software following a (user) misconfiguration of the log4j properties

      Issues: CT-1810

  • Other Issues

    • IPv6 host addresses are now fully supported.

      Can be enabled with the following configuration property:

      prefer-ip-stack="6"

      By default, IPv4 is enabled, which equates to the value of "4" in the above property.

      Issues: CT-1537

Bug Fixes

  • Installation and Deployment

    • When services are deployed with systemd and MySQL could not start due to an error, tpm would not be able to later start MySQL

      Issues: CT-1734

    • deployall script now properly displays the executable prefix for restarting services, typically mm_treplicator in Multi-Site/Active-Active replicator-only installs, as well as the full path to component binaries

      Issues: CT-1835

  • Command-line Tools

    • When MySql services where badly installed, some distribution could show a “not-found” status within systemctl, confusing tpm

      Issues: CT-1677

    • The tpm command now communicates properly when there is no INI configuration file or staging-method deploy.cfg configuration defined.

      Issues: CT-1712

    • The tpm diag command now handles Multi-Site/Active-Active topologies better.

      Issues: CT-1718

    • tungsten_monitor.rb script no longer uses sudo to send emails if the configuration doesn't allow it.

      Issues: CT-1737

    • The tpm update command no longer aborts during a staging deployment when the actual hostname does not match what is configured for the node names in tpm

      Issues: CT-1791

    • Improves the tpm diag command so that it waits 2 seconds for replicator thread dump to complete.

      Issues: CT-1792

    • The tpm connector command now properly passes the -e arguments to the mysql command.

      Issues: CT-1816

    • tpm now properly reports errors upon timeout executing commands. Typically, when a host is down, when upgrading or installing, tpm will now properly report that pinging this host failed.

      Issues: CT-1819

  • Backup and Restore

    • Fixed an issue with latest xtrabackup 8.0.28

      Issues: CT-1838

  • Core Replicator

    • When connecting to a THL server, a client will now connect to the next available host in its THL uri, if the first does not have the sequence number that the client requires. The client will then fail only if none of the hosts from the uri can provide the needed sequence number.

      Issues: CT-1558

    • Fixed an issue when using Parallel apply that would show a NullPointerException in case an event could either not be found or be corrupted in THL. This will now display a correct message Missing or corrupted event from storage

      Issues: CT-1722

    • Fixed an issue where trepctl was leaving JMX connections opened.

      Issues: CT-1752

    • Added more debug information for detecting possible hanging connections while a THL client connects to the THL server. Also, added socket timeout for the connection initialization

      Issues: CT-1760

    • Fixed the EnumToString and pkey filters to renew their database connections (every hour by default). This can be changed with the following properties:

      property=replicator.filter.enumtostring.reconnectTimeout=3600
      property=replicator.filter.pkey.reconnectTimeout=3600

      Issues: CT-1786

  • Tungsten Connector

    • By default, the connector will no longer transparently reconnect underlying connections to database servers when the data service changes.

      This will prevent the following case: in Composite Active/Active topologies, a given connection starts to write data to a site. The site fails, connection gets reconnected the other site and resumes writing. However the data written to the 1st site has not reached the 2nd site, thus data will not be consistent.

      Default is to reject reconnections that follow a write operation (RW_STRICT connection or SmartScale after a write) and to allow reconnection after a read operation (RO_RELAXED or SmartScale after a read) which translates to --connector-allow-cross-site-reconnects-for-writes=false and --connector-allow-cross-site-reconnects-for-reads=true

      It is still possible to get the previous behavior (reconnecting transparently connections cross-site) by specifying both --connector-allow-cross-site-reconnects-for-writes=true and --connector-allow-cross-site-reconnects-for-reads=true, at your own risk

      Issues: CT-1265

    • Connector now mirrors the MySQL default connect_timeout by retrieving it from the primary when starting up. This timeout will apply to all connections made from the connector to MySQL servers.

      This setting can be over-ridden by using the following tpm property

      property=connectTimeout=VALUE

      If VALUE set to autodetect, this value will mirror the MySQL connect_timeout system variable. Set to 0 for infinite timeout.

      Issues: CT-1726

1.15. Tungsten Clustering 6.1.16 GA (20 Dec 2021)

Version End of Life. 15 Aug 2024

Release 6.1.16 contains a number of minor bug fixes and improvements.

Behavior Changes

The following changes have been made to Tungsten Cluster and may affect existing scripts and integration tools. Any scripts or environment which make use of these tools should check and update for the new configuration:

  • Command-line Tools

    • The Perl-based command-line tools now include the latest version of JSON::PP

      Issues: CT-1679

    • The tungsten_show_processlist command has been re-located to the cluster-home/bin from tools/.

      Issues: CT-1695

  • Documentation

    • Updated the open_source_licenses.txt file to include most up-to-date information for the included Perl modules.

      Issues: CT-1675

Improvements, new features and functionality

  • Command-line Tools

    • A new command, tungsten_merge_logs has been added which will gather Manager, Replicator and Connector logs into one file sorted by timestamp for easier troubleshooting.

      Issues: CT-1667

    • A new command tungsten_get_mysql_datadir has been added which will gather and display the actual running value for the mysql data directory (datadir) with any sym-links to the physical destination resolved.

      Issues: CT-1690

Bug Fixes

  • Command-line Tools

    • Now properly adds executable-prefix (when specified) to systemd boot scripts for the replicator.

      Issues: CT-1663

    • The cluster-specific check_tungsten_online tool now properly checks the Replicator and validates the service name if provided.

      Issues: CT-1674

    • The tpm ask tool now works properly on all supported Linux distros.

      Issues: CT-1682

    • The tpm diag command now uses sudo to read the mysqld.log file.

      Issues: CT-1685

    • The tpm --help text now includes the --ini argument.

      Issues: CT-1693

  • Core Replicator

    • Fixes an issue where the replicator would loop forever if a wrong startup option was provided, e.g replicator start wrong

      Now, the replicator will stop with a fatal error message logged Unrecognized option: wrong - Shutting down

      Issues: CT-1672

    • Fixes a replication issue with some JSON fields that would contain a single quote within the JSON text

      Issues: CT-1691

  • Tungsten Connector

    • connector reconfigure, which allows seamless reconfiguration of the Tungsten Connector during tpm update, now properly reloads load balancers.

      Issues: CT-1655

  • Documentation

    • The documentation for the tpm command now describes the use of the --ini argument fully.

      Issues: CT-1692

1.16. Tungsten Clustering 6.1.15 GA (19 Oct 2021)

Version End of Life. 15 Aug 2024

Release 6.1.15 contains a number of minor bug fixes and improvements.

Behavior Changes

The following changes have been made to Tungsten Cluster and may affect existing scripts and integration tools. Any scripts or environment which make use of these tools should check and update for the new configuration:

  • Backup and Restore

    • Cluster backup now uses --source-data as mysqldump parameter from MySQL 8.0.26+

      Issues: CT-1635

Bug Fixes

  • Command-line Tools

    • The tpm update command now handles the --replace-release argument properly when --ini is also specified.

      Issues: CT-1623

    • Added missing connector capability flag CLIENT_PS_MULTI_RESULTS allowing multiple result sets to be retrieved when executing multiple select prepared statements.

      Issues: CT-1624

    • Improved Perl-based command-line tools to use routines with no external dependencies in place of Data::Dumper

      Issues: CT-1630

  • Backup and Restore

    • tungsten_provision_slave would fail using mysqldump with MySQL version 8.0.26+ due to warning of using --master-data=2

      Issues: CT-1612

  • Core Replicator

    • Improved error reporting when the replicator is not able to open a file. This was causing a NullPointerException to be raised.

      Issues: CT-1618

  • Filters

    • Fixed an issue with the ColumnNameFilter that was not resetting its internal connection to the database.

      This would then fail if the database connection would time out (e.g. MySQL wait_timeout reached). The connection is now renewed every 1 hour by default.

      This can be reconfigured to a different value by adding the following tpm property:

      property=replicator.filter.colnames.reconnectTimeout=<time in seconds>

      Issues: CT-1646

  • Tungsten Connector

    • The show slave status hook could use the wrong tracking schema in Composite Active/Passive clusters.

      Issues: CT-1625

1.17. Tungsten Clustering 6.1.14 GA (17 Aug 2021)

Version End of Life. 15 Aug 2024

Release 6.1.14 contains a numebr of bug fixes and improvements in the Connectors, and improved handling of systemd integration, additionally XA transaction support is now available.

Improvements, new features and functionality

  • Command-line Tools

    • Tungsten Connector upgrades are now performed with minimal downtime thanks to graceful-shutdown which is now also used when systemd services are installed.

      systemd definition file has been updated to prevent unwanted systemd-driven control of Tungsten components. 2 new settings have been added to definition files:

      • RemainAfterExit=yes so that systemd won't interfere when the binary is restarted after upgrade

      • GuessMainPID=no so that the PID can change without confusion inside systemd status

      Note that Tungsten has its own service manager (Tanuki service wrapper), systemd is only used for system startup launch of Tungsten components.

      Note

      After upgrade, the systemd status output might show "Active: active (exited)". This is expected since systemd doesn't know the new process to monitor after upgrade

      Issues: CT-1538

    • Added tpm support for tuning maxDelayInterval, used to tune parallel apply. Usage example:

      property=replicator.store.parallel-queue.maxDelayInterval=120

      This sets the max delay interval to 2 minutes

      Issues: CT-1541

    • A new command-line tool tungsten_show_processlist has been added. This is a read-only script which will show the output from tungsten show processlist; on all available Connectors.

      Issues: CT-1542

    • 3 new tpm flags have been added:

      • svc-systemd-config-connector

      • svc-systemd-config-replicator

      • svc-systemd-config-manager

      These flags can be used to provide a custom systemd configuration that will be used in place of the default, generated one.

      This is typically useful when ExecStartPre or ExecStartPost commands are needed

      Issues: CT-1555

    • Improved the tpm command's internal logic and introduced a new sub-command, tpm ask.

      Issues: CT-1573

    • Improved the check_tungsten_online command:

      • determine the cluster name by default if none is provided via -s

      • always check ls resources even if a dataservice name is provided

      • add an option to be node-specific, ignoring any status from other nodes

      Issues: CT-1578

    • New ddlscan template (ddl-mysql-staging.vm) available to generate ddl required for staging tables when deploying the batch applier on a MySQL target.

      Issues: CT-1587

  • Backup and Restore

    • A new tpm option has been added --backup-options

      This property can be used to pass options directly to the backup binary (Xtrabackup, mariabackup etc). This can be especially useful when the xtrabackup and MySQL versions differ. Generally, MySQL is always ahead of xtrabackup and in some situations xtrabackup will fail if the underlying MySQL patch release does not match the xtrabackup patch release. In this scenario, you could utilise the new option to force xtrabackup to not check the versions, for example:

      backup-options=--no-server-version-check

      Multiple options can be passed as a space separated list, for example.

      backup-options=--option1 --option2 --option3

      Note

      Options passed are specific to the underlying binary and are not validated by Tungsten, therefore you must ensure you only use options that are valid, and check syntax accurately.

      Issues: CT-1595

  • Core Replicator

    • Support for XA transactions has been added.

      Issues: CT-925

  • Tungsten Manager

    • A new tpm option has been added replica-tx-commit-level

      Valid values of either 1 (default) or 2. Setting this value to 2 will change the underlying database property innodb_flush_log_at_trx_commit when the node becomes a replica, resetting it back to 1 if the node is promoted to a primary.

      This can improve performance on the apply rates for replicas.

      Issues: CT-1241

Bug Fixes

  • Command-line Tools

    • tpm was failing to check application password when single quoted.

      Issues: CT-1257

    • The --host argument to tpm connector now works properly. Support for the legacy --hosts is also included.

      Only one hostname may be specified to tpm connector.

      Issues: CT-1543

    • The tpm update command now properly parses the --ini argument.

      Issues: CT-1551

    • The tungsten_generate_haproxy_for_api and tpm generate-haproxy-for-api command now properly skips sub-services in Composite Active/Active clusters.

      Issues: CT-1552

    • The tpm connector --samples command no longer displays the password in any example.

      Issues: CT-1593

    • The tpm connector command now works when the application user's password contains double quotes.

      Issues: CT-1596

    • The check_tungsten_services -r command no longer takes an extended time to complete when the manager is not running.

      Issues: CT-1599

  • Tungsten Connector

    • With smartscale enabled, when no writes have been issued to the connection, the user session will remain uninitialized. In that situation, we now send reads to replicas.

      Issues: CT-1560

    • When an RO_RELAXED connection is requested and no replica is available to serve the request, we will now properly reuse the current connection to the primary for executing the request.

      Issues: CT-1564

1.18. Tungsten Clustering 6.1.13 GA (27 May 2021)

Version End of Life. 15 Aug 2024

Release 6.1.13 contains a number of bug fixes, and contains a new Connector feature: Support for Proxy Protocol v1 (if supported by your underlying MySQL release)

Behavior Changes

The following changes have been made to Tungsten Cluster and may affect existing scripts and integration tools. Any scripts or environment which make use of these tools should check and update for the new configuration:

  • Command-line Tools

    • The tpm diag command now gathers tungsten-manager/log/console.log* files.

      Issues: CT-1500

    • The tpm diag command now gathers the tungsten-manager/log/cctrl.log file.

      Issues: CT-1510

    • Ensure that all tools are available in the appropriate software packages.

      Issues: CT-1518

  • Tungsten Connector

    • In a Composite Active/Active topology, connectors within both sites have read and write affinity set to the local site by default.

      Issues: CT-1502, CT-1505

Improvements, new features and functionality

  • Command-line Tools

    • A new tpm option delete-service is now available to simplify the removal of clusters and/or replicator services.

      Issues: CT-210, CT-327, CT-1275

    • A new utility, tungsten_skip_seqno has been added which allows events to be skipped based on filters, allowing the Tungsten Replicator to come back online with less manual intervention.

      Issues: CT-42, CT-1087, CT-1453, CT-1511

  • Tungsten Connector

    • Full support for proxy-protocol v1 has been added to Tungsten Connector.

      Proxy protocol specifications can found here: http://www.haproxy.org/download/1.8/doc/proxy-protocol.txt

      Enabling proxy-protocol:

      • You first need a compatible server, like MariaDB 10.3+ or Percona-5.6.25-73.0+, with the proxy protocol functionality enabled, for example in my.cnf: proxy-protocol-networks=*

      • Then install or update Tungsten with --connector-enable-proxy-protocol=true

      You should now see the origin IP of the connecting application in place of the connector IP in show processlist and tungsten show processlist commands.

      Furher reading:

      Issues: CT-1533

Bug Fixes

  • Command-line Tools

    • This release includes a wide range of improvements and bug fixes for tpm and other command-line tools.

      Issues: CT-1509

    • Fixes an issue where the undeployall command would create a root owned wrapper.log in the ./tools directory.

      Issues: CT-1516

    • Fixed a tpm call to systemd services listing which was making tpm wrongly fall back to init.d. This issue was found when testing MariaDB 10.0.

      Issues: CT-1524

    • tpm now supports new MariaDB my_print_defaults flag --defaults-file

      Issues: CT-1528

  • Core Replicator

    • Fixes an issue where null values in JSON structures were incorrectly quoted once replicated.

      For example :

      [{"example":null}]

      would be replicated as

      [{"example":"null"}]

      Issues: CT-1523

  • Tungsten Connector

    • Now properly supporting negative write affinity in Composite Active/Active clusters. For example you can now specify affinity=localSite,-remoteSite:localSite This allows access only to local sites for writes but still allows reads from remote sites if the local one goes down.

      Issues: CT-1507

  • Tungsten Manager

    • Fixes an issue in Composite Active/Active environments that would occasionally cause the primary node to be set to read only.

      Issues: CT-1424

1.19. Tungsten Clustering 6.1.12 GA (8 Mar 2021)

Version End of Life. 15 Aug 2024

Release 6.1.12 is a minor bug fix release.

Improvements, new features and functionality

  • Command-line Tools

    • A new --no-routers cli option has been added to the cctrl command.

      Using cctrl --no-routers will suppress the collection and display of connectors when issuing the ls command. This is especially useful when there are a lot of connectors installed and gathering the information from them could take a lot of time, causing unexpected timeouts.

      Issues: CT-1437

  • Tungsten Connector

  • Tungsten Manager

    • Minor improvements to rule handling in the managers.

      Issues: CT-1464

  • Monitoring

    • Support has now been added to allow monitoring via the New Relic Insights API, using a new script tungsten_newrelic_event

      Issues: CT-892

Bug Fixes

  • Command-line Tools

    • If using mysqldump as your preferred choice of backup method, Stored Procedures and Functions would not be included in the generated dump file when using tungsten_provision_slave.

      Issues: CT-1339

    • In a Composite Active/Active environment, the tungsten_post_process command now intelligently handles cross-site filter properties in the INI so that any defined filters are appended to the default values for any option that resolves to property replicator.stage.q-to-dbms.filters

      Issues: CT-1372, CT-1449

    • Replaced md5 hashing to sha256 within tpm to fix issues where tpm install would fail on FIPS enabled environments.

      Issues: CT-1451

    • When provisioning from a Primary node, in a Composite Active/Active environment, using tungsten_provision_slave, the replicator would fail to reset due to the incorrect service name being used.

      Issues: CT-1458

    • Fixed an issue where, in Composite Active/Active environments, tpm would ignore the affinity passed with the tpm flag --connector-affinity.

      Issues: CT-1465

    • If using tungsten_provision_slave to restore a previously failed Primary node, the provision could fail to recover the node.

      Issues: CT-1468

  • Core Replicator

    • A new mapcharset filter is included, which fixes collation incompatibilities when replicating from MySQL 8 to an older release (5.7 or 5.6)

      This new filter should be enabled within any MySQL applier where the source is generated from a MySQL 8 database and the target is MySQL 5.7 or lower.

      Issues: CT-1404

    • If SSL is enabled between replicators for THL Transfer, and a source node fails and becomes unresponsive, or is slow/overloaded, the replicator on the replica can fail to go offline due to a timeout, remaining in a GOING-ONLINE:SYNCHRONIZING state.

      In a clustering envronment, this can prevent failovers occuring due to the manager waiting for the replicator which is stuck in a timeout loop.

      Issues: CT-1459

1.20. Tungsten Clustering 6.1.11 GA (21 Jan 2021)

Version End of Life. 15 Aug 2024

Release 6.1.11 is a critical bug fix addressing an issue found in the Managers.

Bug Fixes

  • Tungsten Manager

    • Fixed a bug in the Hedera library that prevented Managers to pass through the init state.

      Issues: CT-1415

1.21. Tungsten Clustering 6.1.10 GA (15 Dec 2020)

Version End of Life. 15 Aug 2024

Release 6.1.10 is a critical bug fix addressing an issue with Parallel Apply within a clustered environemnt.

Improvements, new features and functionality

  • Command-line Tools

    • tpm diag now collects routing table information via route -n , and has two new command-line arguments: --include and --groups.

      --include specifies a comma-separated list of subroutines to include. Any gather subroutine not listed will be skipped.

      --groups specifies a comma-separated list of subroutine groups to include. Any group not listed will be skipped.

      Issues: CT-1399

  • Filters

    • A number of sql_modes have been removed in MySQL 8.0, meaning that when replicating from an earlier version (e.g. 5.7) the applier could error when applying session variables.

      A new dropsqlmode filter is now available that should be configured against appliers where the source is version 5.7 or earlier and the target is version 8.0 or later.

      Issues: CT-1361

Bug Fixes

  • Tungsten Manager

    • If parallel apply has been enabled, automatic failovers could hang indefinetly if one of more channels are inactive.

      Important

      All customer using parallel apply are urged to apply this release to avoid production outages due to this known issue.

      Issues: CT-1407

1.22. Tungsten Clustering 6.1.9 GA (23 Nov 2020)

Version End of Life. 15 Aug 2024

Release 6.1.9 is a minor bug fix release containing a fix for a critial bug within the Replicator related to the handling of Timezone.

Improvements, new features and functionality

  • Command-line Tools

    • A new script is available, tungsten_generate_haproxy_for_api. This script will read all available INI files and dump out corresponding haproxy.cfg entries with properly incrementing ports; the composite parent will come first, followed by the composite children in alphabetical order.

      This script will be embedded as a tpm command in a future release.

      Issues: CT-1385

Bug Fixes

  • Command-line Tools

    • tpm update no longer fails when using the staging method to upgrade to a new version.

      Issues: CT-1381

    • tungsten_find_orphaned no longer hangs when the password keyword exists by itself under [client] in my.cnf, which caused mysqlbinlog to issue a password prompt.

      Issues: CT-1387

  • Core Replicator

    • In some edge case scenarios, the replicator was not setting the session time_zone correctly when proceeding sessions had a different time_zone applied, this could lead to situations where TIMESTAMP values could be applied into replica nodes with an incorrect time_zone offset applied.

      Issues: CT-1390

1.23. Tungsten Clustering 6.1.8 GA (2 Nov 2020)

Version End of Life. 15 Aug 2024

Release 6.1.8 is a minor bug fix release.

Bug Fixes

  • Command-line Tools

  • Core Replicator

    • Fixes an issue that would prevent a service from going offline at a specified time (trepctl online -until-time) or at a specific seqno (trepctl online -until-seqno) when parallel apply is enabled.

      Issues: CT-1243

    • In MySQL releases using old row events format (MySQL 5.6 or earlier), Delete_rows_v1 were badly handled, leading to an extraction error when handling such an event type.

      Issues: CT-1358

  • Tungsten Manager

    • Managers now accept both unencrypted and TLS connections from connectors. This allows migrations when security is enabled from earlier versions to 6.1.2+, as well as seamless switch to encrypted connector-manager communications.

      Important

      When upgrading to this release from an earlier release, and SSL is either already enabled, or being enabled as part of the upgrade, it is important that managers are upgraded before the connectors.

      Note that non-secured connectors connections will trigger a warning in the manager logs:

      WARN [RouterGatewayServer] - Un-encrypted connection request from <connector address> while this manager »
      is configured to use SSL. This is expected ONLY when migrating manager<>connector communication to SSL. »
      Otherwise it might be a misconfiguration of this connector.

      Issues: CT-1320

    • A network partition between primary and relay sites with port 11999 still functioning was leading to full connectivity outage. Managers now send data services configuration with every node update (one per node every 3s) so that connectors remain up-to-date even in that corner case.

      Issues: CT-1356

    • Fixes a memory leak when non recognized connections to manager port 11999 were made.

      Issues: CT-1357

1.24. Tungsten Clustering 6.1.7 GA (5 Oct 2020)

Version End of Life. 15 Aug 2024

Release 6.1.7 was a minor bug fix release containing a fix for SSL Communications specific to clustering deployments.

Bug Fixes

  • Tungsten Connector

    • Disables the naggle algorithm (TcpNoDelay=true) on SSL Sockets which causes performance degredation on SSL communications within the connector.

      Issues: CT-1331

    • Allows configuration of the protocols and cipher suites to use within the drizzle driver for SSL communications to the database server.

      New TPM parameters can be used to control this. New parameters are as follows:

      connector-server-ssl-protocols=protocol1[,protocol2,...]
      connector-server-ssl-ciphers=cipher1[,cipher2,...]

      For example:

      connector-server-ssl-protocols=TLSv1.2
      connector-server-ssl-ciphers=TLS_RSA_WITH_AES_256_CBC_SHA256,TLS_RSA_WITH_AES_256_CBC_SHA512

      Default values for connector-server-ssl-protocols will be TLSv1,TLSv1.1,TLSv1.2. Default values for connector-server-ssl-ciphers will be to allow all cipher suites supported by the running JVM.

      Issues: CT-1335

    • When the Primary node is not available (for example during switch/failover), the connector would pause incoming RO_RELAXED connection requests even if a Replica is available for reads.

      Note

      This only applies to connectors configured in Bridge Mode.

      Issues: CT-1337

    • c3p0 libraries upgraded to version 0.9.5.5 and adjusted recommended pool configuration.

      Issues: CT-1343

1.25. Tungsten Clustering 6.1.6 GA (20 Aug 2020)

Version End of Life. 15 Aug 2024

Release 6.1.6 is a minor bug fix containing a fix for Composite Active/Active environments.

Bug Fixes

  • Command-line Tools

    • tpm install and tpm update will now call tungsten_post_process automatically to ensure cross-site specific configurations are applied at the right time during the install and update process within Composite Active/Active installations.

      Issues: CT-761

  • Core Replicator

    • Allows multiple service names to be supplied to property=local.service.name when configuring bi-directional replication between a Composite Active/Passive cluster topology and a MySQL target to prevent loopback of transactions.

      Issues: CT-1308

1.26. Tungsten Clustering 6.1.5 GA (5 Aug 2020)

Version End of Life. 15 Aug 2024

Release 6.1.5 is a small interim bug fix with a number of issues resolved within the Core Replicator, and a specific issue within Tungsten Connector in Multi-Cluster Environments.

Known Issue

The following issues are known within this release but not considered critical, nor impact the operation of Tungsten Cluster. They will be addressed in a subsequent patch release.

  • Core Replicator

    • In MySQL release 8.0.21 the behavior of CREATE TABLE ... AS SELECT ... has changed, resulting in the transactions being logged differenly in the binary log. This change in behavior will cause the replicators to fail.

      Until a fix is implemented within the replicator, the workaround for this will be to split the action into separate CREATE TABLE ... followed by INSERT INTO ... SELECT FROM... statements.

      If this is not possible, then you will need to manually create the table on all nodes, and then skip the resulting error in the replicator, allowing the subsequent loading of the data to continue.

      Issues: CT-1301

Bug Fixes

  • Core Replicator

    • When replicating data that included timestamps, the replicator would update the timestamp value to the time within the commit from the incoming THL. When using statement based replication times would be correctly replicated, but if using a mixture of statement and row based replication, the timestamp value would not be set back to the default time when switching between statement and row based events. This would not cause problems in the applied host, except when log_slave_updates was enabled. In this case, all row-based events after a statement based event would have the same timestamp value applied.

      This was most commonly seen when using the standalone replicator to replicate into a Cluster, either from a standlone source, or a cluster source.

      Issues: CT-1255

    • If filtering is in use, and a space appeared either side of the delimiter in a "schema.table" definition in your SQL, the replicator would fail to parse the statement correctly.

      For example, all of the examples below are valid SQL but would cause a failure in the replicator:

      sql> CREATE TABLE myschema. mytable (....
      
      sql> CREATE TABLE myschema .mytable (....
      
      sql> CREATE TABLE myschema . mytable (....

      Issues: CT-1278

    • Fixes a bug in the Drizzle Driver whereby a failing prepared statement that exceeds 1000 characters would report a String index out of range: 999 error rather than the actual error.

      Issues: CT-1303

  • Tungsten Connector

    • Connector would fail to properly find appropriate data service for its auto-configuration. Now using the service provided in the user map when present.

      Note

      This fix is also available in build 51 of the 6.1.4 release.

      Issues: CT-1277

1.27. Tungsten Clustering 6.1.4 GA (4 June 2020)

Version End of Life. 15 Aug 2024

Release 6.1.4 contains a number of improvements and bug fixes, specifically for the tpm command line tool and stability improvements for Composite Active/Active topologies. In addition this release now fully supports the latest binlog compression feature of MySQL 8.0.20.

Improvements, new features and functionality

  • Tungsten Manager

    • When a Primary is OFFLINE or SHUNNED, we no longer mark the whole data service as OFFLINE.

      Note

      This will allow reading from Replicas to continue even if the Primary is offline.

      Issues: CT-1152

    • In a Composite Active/Active topology, the relay hosts, by default, will pull THL from the remote Primaries. This can be changed to pull from remote Replica(s) by use of the following setting policy-relay-from-slave=true

      Note

      This is a change in behaviour. From version 6.0.4 up to and including 6.1.3, the default behaviour was configured to pull from the remote Replica(s).

      Issues: CT-1211

Bug Fixes

  • Command-line Tools

    • The tpm diag command now captures cctrl ls output from the Composite dataservice when appropriate.

      Issues: CT-1139

    • Fixes an issue where affinity specified in the configuration (connector-affinity) was ignored since it was not written to user.map (this affects proxy configurations only).

      Issues: CT-1146

    • Fixed a bug where tpm diag would fail to gather some MySQL information on a Composite Active/Active node.

      Issues: CT-1167

    • Fixed a bug in tungsten_post_process where the /etc/tungsten/tungsten.ini file would not be read properly. Also adds two warnings for skipped entries and corrects a typo in the property definitions.

      Issues: CT-1198

  • Tungsten Connector

    • Fixed an issue where connector client-list and the Proxy-mode command, tungsten show processlist, would report NullPointerException errors when listing disconnected client applications.

      Issues: CT-1177

    • When in proxy mode and fetching data source configuration at startup, the Connector will now properly parse, and use, affinity (and other URL options if any) to establish the connection.

      Issues: CT-1212

    • Fixes an issue where connections with multiple affinity would not keep the current channel and reconnect to the database server while the connection could be reused.

      Issues: CT-1250

  • Tungsten Manager

    • In certain edge case situations, the manager could choose a manually shunned node as a viable node during failover.

      Issues: CT-1115

    • Issuing cluster topology validate within cctrl would fail if the cluster contained an Active Witness host.

      Issues: CT-1180

    • Piping multiple commands to cctrl that would affect components in a remote cluster would fail, for example:

      shell> echo "use east; replicator db5 offline" | cctrl

      Issues: CT-1209

    • Resolves an edge case in a Composite Active/Passive topology, with 2 or more Composite Passive dataservices, where a switch of a relay node in a single Replica service would incorrectly reconfigure all relays in the other Replica services.

      Issues: CT-1214

Tungsten Clustering 6.1.4 Includes the following changes made in Tungsten Replicator 6.1.4

Release 6.1.4 contains a number of improvements and bug fixes, specifically for the tpm command line tool and improvements to the Redshift Applier. In addition this release now fully supports the latest binlog compression feature of MySQL 8.0.20.

Improvements, new features and functionality

  • Command-line Tools

    • Improves tpm performance by using more efficient routines to calculate paths.

      Issues: CT-1168

    • Added the ability for tpm diag to skip both individual gather subroutines along with entire groups of gather subroutines.

      Also added ability to list all gather groups and subroutines using --list for use with the --skip and --skipgroups cli arguments.

      Issues: CT-1176

    • tungsten_provision_slave has been rewritten fixing a number of issues in the previous release. This version was previously released as the Beta script prov-sl.sh.

      Issues: CT-1208

    • A number of performance improvements and fixes have been incorporated into the Drizzle Driver used for communication between components and MySQL, these include:

      • Better handling of large queries close to max network packet size.

      • Batch Support. Instead of sending statements one by one, the driver will be able to send multiple statements at once, avoiding round trips between the driver and MySQL server.

      • Fixes issues with interpreting useSSL on connect string URLs.

      Issues: CT-1215, CT-1216, CT-1217, CT-1228

    • The tungsten_send_diag command now accepts arguments for the tpm diag command using --args or -a for short.

      You must enclose the arguments in quotes, for example:

      shell> tungsten_send_diag -c 9999 -d --args ‘--all -v’

      Issues: CT-1220

  • Core Replicator

    • debug has been disabled by default in the schemachange filter. Resulting in reduced noise in the replicator log file.

      Issues: CT-1013

    • A new replicator role thl-client has been added. This new role allows the replicator to download THL from a Primary, but not apply to the target database.

      Issues: CT-1123

    • A new delayInMs filter has been added which allows the applying of THL to a Replica to be delayed. The filter allows millisecond precision. This filter works in the same way as the TimeDelayFilter, however that filter only allow second precision.

      Issues: CT-1191

    • A new droprow JavaScript filter has been added.

      The filter works at ROW level and allows the filtering out of rows based on one or more column/value matches

      Issues: CT-1213

    • When configuring the Redshift applier, you can now configure which tool the applier will use for posting CSV files to S3. Options are s3cmd (default), s4cmd or aws.

      Issues: CT-1218

    • A number of improvements have been made to the Redshift Applier to allow optional levels of table locking.

      This is particularly useful when you have multiple Redshift Appliers in a Fan-In topology, and/or very high volumes of data to process.

      The additional locking options reduce the risk of Redshift Serializable Isolation Violation errors occuring.

      Full details of how to utilise the new options can be read at Handling Concurrent Writes from Multiple Appliers

      Issues: CT-1221

    • Tungsten Replicator can now correctly extract and parse Binary Log entries when the MySQL option binlog-transaction-compression has been enabled.

      binlog-transaction-compression is a new parameter introduced from MySQL 8.0.20.

      Issues: CT-1223

Bug Fixes

  • Command-line Tools

    • In certain edge cases, tungsten_provision_slave would fail to detect if mysql was shutdown.

      Issues: CT-1096

    • tpm diag now collects directories specified with !includedir in the /etc/my.cnf file.

      Issues: CT-1153

    • Fixes the tpm update command, which would exit with the error:

      Argument " (error code 1)" isn't numeric

      Issues: CT-1165

    • tpm diag now collects any files specified by !include directives in the /etc/my.cnf and /etc/mysql/my.cnf files.

      tpm diag also looks in /etc/mysql/my.cnf for !includedir directives

      Issues: CT-1169

    • Fixes a bug which prevented tungsten_send_diag from uploading a self-generated diagnostic zip file.

      Issues: CT-1170

    • tpm diag now properly derives the correct target path to the releases directory if the home directory in the configuration points to a sym-link.

      Issues: CT-1172

    • Removed tpm diag call to sudo for gathering ifconfig and lsb_release commands.

      Issues: CT-1175

    • tpm update would fail and report errors if deployall had been executed.

      Issues: CT-1179

    • tpm diag no longer requires the mysql command-line client when running on non-MySQL Applier nodes, and no longer attempts to gather any MySQL diagnostic information.

      Issues: CT-1188

    • Removes the requirement to execute component start/stop commands with sudo when called through systemd.

      This is specific to start/stop actions following the use of the deployall scripts.

      Issues: CT-1193

    • Fixes cases where tpm fails when the OS hostname command returns a different string than what is used in the configuration (i.e. hostname returns a FQDN, yet the configuration contains shortnames like db1, db2, etc.).

      Issues: CT-1206

    • In certain cases, after a reprovision, tungsten_provision_slave didn’t always run the steps to reset the local replicator service. This made the replicator go into an error state after provision had completed.

      Issues: CT-1210

    • The tpm diag command now handles the cluster-slave topology more gracefully, and properly handles cluster nodes without the Connector installed.

      Improved output text clarity by converting multiple verbose-level outputs to debug, and warnings to notice-level.

      Issues: CT-1222

    • tpm diag now gathers sym-linked files correctly.

      Issues: CT-1232

    • ddlscan now sets the datatype for sequence number columns to BIGINT when generating staging table DDL for Redshift deployments.

      Issues: CT-1235

    • Fixes a situation where tpm update exits with a Data::Dumper error.

      Issues: CT-1249

  • Core Replicator

    • In heterogeneous replicator deployments, the convertstringfrommysql filter would fail to convert strings for alphanumeric key values.

      Issues: CT-1128

    • Tungsten Replicator could wrongly think it is already in DEGRADED mode when trying to put it into DEGRADED state.

      Issues: CT-1131

    • Tungsten Replicator now recognises Amazon AWS SSL Certificates to enable SSL communication with AWS Aurora.

      Issues: CT-1173

    • When host server time (and thus MySQL time) is not configured as UTC, issuing cluster heartbeat or trepctl heartbeat in the first hours around daylight savings time would create an invalid time in MySQL.

      For more information on timezones when issuing heartbeats, see trepctl heartbeat Time Zone Handling

      Issues: CT-1174

    • The replicator would fail to apply into Vertica when configured as an offboard installation due to the applier incorrectly expecting the csv files to exist locally on the remote Vertica host.

      Issues: CT-1194

    • Due to a change in the Binary log structure introduced in MySQL 8.0.16, the replicator would fail to extract transactions for Partitioned tables.

      Issues: CT-1201

    • The replicator would fail to correctly parse statements with leading comment blocks in excess of 200 characters.

      Issues: CT-1236

1.28. Tungsten Clustering 6.1.3 GA (17 February 2020)

Version End of Life. 15 Aug 2024

Release 6.1.3 contains a small number of critical bug fixes that can affect customers operating geo-distributed clusters across high latency network links, along with a small number of improvements and fixes to common command line tools.

Behavior Changes

The following changes have been made to Tungsten Cluster and may affect existing scripts and integration tools. Any scripts or environment which make use of these tools should check and update for the new configuration:

  • Command-line Tools

    • Improved the tungsten_find_position script to add the ability to specify the low and high sequence numbers which limits the amount of THL the script needs to parse. This allows for better performance and lower system overhead. Also allows the use of the --file option.

      [-f|--file] Pass specified file to the thl command as -file {file}
      [--low|--from] Pass specified seqno to the thl command as -low {seqno}
      [--high|--to] Pass specified seqno to the thl command as -high {seqno}

      Issues: CT-1143

    • tpm diag : make the output from remote host diagnostic gathering visible in addition to alerting when a host is not reachable.

      Issues: CT-1158

Known Issue

The following issues are known within this release but not considered critical, nor impact the operation of Tungsten Cluster. They will be addressed in a subsequent patch release.

  • Backup and Restore

    • The backup process, when configured to use Xtrabackup, uses the --stream=tar option as one of the options passed to the backup process.

      This option is no longer available in Xtrabackup 8.0

      If you use Xtrabackup 8.0 in combination with MySQL 8, generating backups using the procedures available in Tungsten Clustering will fail. Until a fix is available and to allow backups to continue you will need to make the following edit to the configuration

      After installation, open the static-servicename.properties file located in INSTALL_PATH/tungsten/tungsten_replicator/conf

      Locate the following entry replicator.backup.agent.xtrabackup.options and within the string value, change the value of tar=true to tar=false

      If the replicator is already running, then you will need to issue replicator restart for the change to take effect

      Warning

      Changing the properties file directly will cause future tpm update commands to fail, therefore you should run this with the --force option, and then redit the file as per the above instructions to reset the tar option

      Issues: CT-1157

Bug Fixes

  • Command-line Tools

    • tpm diag would fail to collect diagnostics for relay nodes within a Composite Active/Passive Topology

      Issues: CT-1140

    • Fixes an edge case whereby tpm mysql would fail on a node within a Composite Active/Active topology.

      Issues: CT-1151

    • tpm diag now gathers all hosts in a staging deployment when run from a non-staging node.

      Issues: CT-1155

    • tpm diag : fix ensures collection of diagnostics from standalone connector hosts.

      Issues: CT-1159

  • Tungsten Manager

    • During a local Primary switch within a Composite Active/Active Topology, where there is a high latency link between clusters, the switch could intermittently fail due to an incorrect rule triggering as the remote cluster sees an incorrect state in the opposing cluster

      Issues: CT-1141

Tungsten Clustering 6.1.3 Includes the following changes made in Tungsten Replicator 6.1.3

Release 6.1.3 contains a small number of improvements and fixes to common command line tools, and introduces compatibility with MongoDB Atlas.

Behavior Changes

The following changes have been made to Tungsten Replicator and may affect existing scripts and integration tools. Any scripts or environment which make use of these tools should check and update for the new configuration:

  • Command-line Tools

    • tpm diag has been updated to provide additional feedback detailing the hosts that were gathered during the execution, and also provides examples of how to handle failures

      When running on a single host configured via the ini method:

      shell> tpm diag
      Collecting localhost diagnostics.
      Note: to gather for all hosts please use the "-a" switch and ensure that you have paswordless »
      ssh access set between the hosts.
      Collecting diag information on db1.....
      Diagnostic information written to /home/tungsten/tungsten-diag-2020-02-06-19-34-25.zip

      When running on a staging host, or with the -a flag:

      shell> tpm diag [-a|--allhosts]
      Collecting full cluster diagnostics
      Note: if ssh access to any of the cluster hosts is denied, use "--local" or "--hosts=<host1,host2,...>"
      Collecting diag information on db1.....
      Collecting diag information on db2.....
      Collecting diag information on db3.....
      Diagnostic information written to /home/tungsten/tungsten-diag-2020-02-06-19-34-25.zip

      Issues: CT-1137

Bug Fixes

  • Command-line Tools

    • tpm would fail to run on some Operating Systems due to missing realpath

      tpm has been changed to use readlink which is commonly installed by default on most operating systems, however if it is not available, you may be required to install GNU coreutils to satisfy this dependency

      Issues: CT-1124

    • Removed dependency on perl module Time::HiRes from tpm

      Issues: CT-1126

    • Added support for handling missing dependency (Data::Dumper) within various tpm subcommands

      Issues: CT-1130

    • tpm will now work on MacOS/X systems, provided greadlink is installed.

      Issues: CT-1147

    • tpm install will no longer report that the linux distribution cannot be determined on SUSE platforms.

      Issues: CT-1148

    • Fixes a condition where tpm diag would fail to set the working path correctly, especially on Debian 8 hosts.

      Issues: CT-1150

    • tpm diag now checks for OS commands in additional paths (/bin, /sbin, /usr/bin and /usr/sbin)

      Issues: CT-1160

    • Fixes an issue introduced in v6.1.2 where the use of the undeployall script would stop services as it removed them from systemctl control

      Issues: CT-1166

  • Core Replicator

    • The MongoDB Applier has been updated to use the latest MongoDB JDBC Driver

      Issues: CT-1134

    • The MongoDB Applier now supports MongoDB Atlas as a target

      Issues: CT-1142

    • The replicator would fail with Unknown column '' in 'where clause when replicating between MySQL 8 hosts where the client application would write data into the source database host using a different collation to that of the default on the target database.

      The replicator would fail due to a mismatch in these collations when querying the information_schema.columns view to gather metadata ahead of applying to the target

      Issues: CT-1145

1.29. Tungsten Clustering 6.1.2 GA (20 January 2020)

Version End of Life. 15 Aug 2024

Release 6.1.2 contains significant improvements as well as some needed bugfixes.

Behavior Changes

The following changes have been made to Tungsten Cluster and may affect existing scripts and integration tools. Any scripts or environment which make use of these tools should check and update for the new configuration:

  • Behavior Changes

    • The Passive Witness functionality is now officially DEPRECATED and will be REMOVED in version 6.2

      Issues: CT-653

  • Installation and Deployment

    • In cluster deployments with witness nodes, if the MySQL servers had been configured to listen on any port other than the standard 3306, the tpm command would default to the wrong port number.

      When witness hosts are in use, the tpm command will no longer "guess" the data source port automatically by running my_print_defaults.

      It is now mandatory to specify the MySQL data server port explicitly using --datasource-port={mysql_listen_port} or one of its aliases

      Issues: CT-1071

Improvements, new features and functionality

  • Command-line Tools

    • The tpm command was originally written in Ruby. This improvement converts tpm to Perl over time, starting with the tpm shell wrapper and refactoring each sub-command one-by-one.

      For this release, we have redone the diag, mysql and connector sub-commands.

      This also wraps the update sub-command to provide the CT-1093 clustering fix.

      Issues: CT-1048

    • This version includes an update to the new BETA tool to provision Replicas, and Primaries in a Composite Active/Active topology. This release fully supports provisioning of nodes in a Composite Active/Active Topology.

      It can be invoked by running prov-sl.sh or tps.pl.

      This tool will replace tungsten_provision_slave in a future release.

      Issues: CT-1070

    • Added the tpm policy subcommand to allow easy get and set cluster policy operations instead of using cctrl.

      For more information, please see ...

      Issues: CT-1106

    • The new changes made to the tpm command, require that the zip package be installed on all DB hosts

      Issues: CT-1111

  • Tungsten Manager

    • The new default datasources sort order is alphabetically when using the cctrl ls command. Additionally, the sort order of the datasources list is now configurable.

      The behavior is controlled by the tpm configuration property cctrl.sort.datasources.alphabetically which has a default value of true (meaning alpha sort).

      If set to false, the sort is ordered by datasource role, so the Primary or relay will appear first, followed by the Replicas. For example, use the following in the INI file for role-based sorting:

      property=cctrl.sort.datasources.alphabetically=false

      Issues: CT-1018

Bug Fixes

  • Installation and Deployment

  • Command-line Tools

    • When performing an update of a cluster with tpm, the cluster would be switched to MAINTENANCE but would remain in this policy after the update. The original policy is now retained during the update.

      Issues: CT-595, CT-1093

    • The deployall script was only able to install init.d system startup scripts.

      In this release, the script will now detect the initialization system in use (systemd or initd) and prefer systemd when both are available.

      For systemd configurations only:

      For continuity of service reasons, the deployall script does NOT restart individual components when called, it will only install systemd scripts. This implies that, right after a call to deployall and before host restart, the system will stay in a mixed mode where systemd scripts are in place but components were started without systemd, so won’t be controllable by it.

      In order to align the configuration, you will need to run

      shell> component stop sysd
      shell> sudo systemctl start tcomponent

      For example:

      shell> connector stop sysd
      shell> sudo systemctl start tconnector

      Issues: CT-853

    • When issuing tpm connector --samples, the output displayed clear text passwords. In this release the passwords are obsfucated.

      Issues: CT-1021

    • Continuent Tungsten Clustering now uses the xtrabackup command instead of the deprecated innobackupex to create and restore backups. A new check was added to TPM for validating different xtrabackup versions along with MySQL version compatibility. The oldest supported version of xtrabackup is v2.3

      Issues: CT-1074

  • Core Replicator

    • When configuring SSL for the Connector only, the Replicator would fail to start due to the Replicator also looking for the SSL configuration.

      Issues: CT-956

  • Tungsten Connector

    • Fixed an issue where some applications might fail to connect to the Connector with MariaDB 10+

      Previously, when using MariaDB 10+, the Connector would be confused by the 10 and will think it is a MySQL 8+ server. By default, the Connector will offer to connect with caching_sha2_password. If the application does not know how to switch authentication plugins, it would fail with a message similar to the following:

      The server requested authentication method unknown to the client [caching_sha2_password]

      The previous work-around was to specify the authentication plugin using the tpm command:

      --property=defaultAuthPlugin=mysql_native_password

      Issues: CT-1033

    • Improved Tungsten Connector bridge mode performance when transferring small amounts of data.

      Issues: CT-1081

  • Tungsten Manager

    • When using the cctrl command interactively, the `cluster topology` TAB completion was showing invalid options. Invalid options have been removed.

      Issues: CT-979

    • Fixed an issue where long-duration operations like failover and switch would create false positives about network partitioning after completion.

      Issues: CT-1023

    • Fixed the cctrl command so that the '[SSL]' indicator in the `ls` output is displayed. This is a Version 5 feature that was lost in v6.0.0, now restored.

      Issues: CT-1061

    • Fixed the cctrl command `datasource {hostname} restore` which was failing in Composite Active/Active cluster deployments with: ERROR: MORE THAN ONE PRIMARY DATA SOURCE FOUND.

      Issues: CT-1062

    • Continuent Tungsten Clustering now only checks for a running MySQL server when the backup method is 'mysqldump' in cctrl.

      Background: Running datasource {hostname} restore inside cctrl would fail when the MySQL server was not running. Only the 'mysqldump' method requires a running MySQL server. The 'xtrabackup-full' and 'xtrabackup-incremental' methods will work even if MySQL is stopped.

      Issues: CT-1077

    • tungsten_find_orphaned was displaying an incorrect error message if a service name wasn't supplied correctly.

      Issues: CT-1079

    • tungsten_find_orphaned would error with 'Argument "" isn't numeric in addition'.

      Issues: CT-1080

    • Fixed an issue where Composite clusters with only a single site would come up as SHUNNED after install.

      Issues: CT-1101

Tungsten Clustering 6.1.2 Includes the following changes made in Tungsten Replicator 6.1.2

Release 6.1.2 contains both significant improvements as well as some needed bugfixes.

Behavior Changes

The following changes have been made to Tungsten Replicator and may affect existing scripts and integration tools. Any scripts or environment which make use of these tools should check and update for the new configuration:

  • Behavior Changes

    • Certified the Tungsten product suite with Java 11.

      A small set of minor issues have been found and fixed (CT-1091, CT-1076) along with this certification.

      The code is now compiled with Java compiler v11 while keeping Java 8 compatibility.

      Java 9 and 10 have been tested and validated but certification and support will only cover Long Term releases.

      Note

      Known Issue

      With Java 11, command line tools are slower. There is no impact on the overall clustering or replication performance but this can affect manual operations using CLI tools such as cctrl and trepctl

      Issues: CT-1052

Improvements, new features and functionality

  • Core Replicator

    • A new Replicator role, thl-server, has been added.

      This new feature allows your Replica replicators to still pull generated THL from a Primary even when the Primary replicator has stopped extracting from the binlogs.

      If used in Tungsten Clustering, this feature must only be enabled when the cluster is in MAINTENANCE mode.

      Issues: CT-58

      For more information, see Understanding Replicator Roles.

    • A new JavaScript filter dropddl.js has been added to allow selective removal of specific object DDL from THL.

      Issues: CT-1092

Bug Fixes

  • Behavior Changes

    • If you need to reposition the extractor, there are a number of ways to do this, including the use of the options -from-event or -base-seqno

      Both of these options are mutually exclusive, however in some situations, such as when positioning against an Aurora source, you may need to issue both of these options together. Previously this was not possible. In this release both options can now be supplied providing that you include the additional -force option, for example

      shell> trepctl -service serviceName online -base-seqno 53 -from-event 000412:762897 -force

      Issues: CT-1065

    • When the Replicator inserts a heartbeart there is an associated timezone. Previously, the heartbeat would be inserted using the GMT timezone, which fails during the DST switch window. The new default uses the Replicator host's timezone instead.

      This defaults change corrects an edge case where inserting a heartbeat will fail during the DST switch window when the MYSQL server is running in a different timezone than the Replicator (which runs in GMT).

      For example, on 31th March 2019, the time switch occurred @ 2AM in the Europe/Paris timezone. When inserting a heartbeat in the window between 4 and 5 AM (say at 4:15am), the corresponding GMT time would be 2:15am, which is invalid in the Europe/Paris timezone. Replicator would then fail if the MySQL timezone was set to Europe/Paris, as it would try to insert an invalid timestamp.

      A new option, -tz has been added into the trepctl heartbeat command to force the use of a specific timezone.

      For example, use GMT as the timezone when inserting a heartbeat:

      shell> trepctl heartbeat -tz NONE

      Use the Replicator host's timezone to insert the heartbeat:

      shell> trepctl heartbeat -tz HOST

      Use the given timezone to insert the heartbeat:

      shell> trepctl heartbeat -tz {valid timezone id}

      If the MySQL server timezone is different from the host timezone (which is strongly not recommended), then -tz {valid timezone id} should be used instead where {valid timezone id} should be the same as the MySQL server timezone.

      Issues: CT-1066

    • Corrected resource leak when loading Java keystores

      Issues: CT-1091

  • Command-line Tools

    • Fixed error message to indicate the need to specify a service on Composite Active/Active clusters for the tungsten_find_position and tungsten_find_seqno commands.

      Issues: CT-1098

    • The tpm command no longer reports warnings about existing system triggers with MySQL 8+

      Issues: CT-1099

  • Core Replicator

    • When configuring a Kafka Applier, the Kafka Port was set incorrectly

      Issues: CT-693

    • If a JSON field contained a single quote, the replicator would break during the apply stage whilst running the generated SQL into MySQL.

      Single quotes will now be properly escaped to solve this issue

      Issues: CT-983

    • Under rare circumstances (network packet loss or MySQL Server hang), the replicator would also hang until restarted.

      This issue has been fixed by using specific network timeouts in both the replicator and in the Drizzle jdbc driver connection logic

      Issues: CT-1034

    • When configuring Active/Active, standalone replicators, with the BidiSlave filter enabled, the replicator was incorrectly parsing certain DDL Statements and marking them as unsafe, as a result they were being dropped by the applier and ignored

      The full list of DDL commands fixed in this release are as follows:

      • CREATE|DROP TRIGGER

      • CREATE|DROP FUNCTION

      • CREATE|DROP|ALTER|RENAME USER

      • GRANT|REVOKE

      Issues: CT-1084, CT-1117

    • The following warnings would appear in the replicator log due to GTID events not being handled.

      WARN extractor.mysql.LogEvent Skipping unrecognized binlog event type 33, 34 or 35)

      The WARN message will no longer appear, however GTID Events are still not handled in this release, but will be fully extracted in a future release.

      Issues: CT-1114

1.30. Tungsten Clustering 6.1.1 GA (28 October 2019)

Version End of Life. 15 Aug 2024

Release 6.1.1 contains both significant improvements as well as some needed bugfixes.

Improvements, new features and functionality

  • Tungsten Manager

    • Improved how the Manager and Replicator behave when MySQL dies on the Primary node.

      This improvement will induce a change of behavior in the product during failover by default, possibly causing a delay in failover as a way to protect data integrity.

      The new default setting for 6.1.1 is:

      replicator.store.thl.stopOnDBError=false

      This means that the Manager will wait until the Replicator reads all remaining binlog events on the failing Primary node.

      Failover will only continue once:

      • all available events are completely read from the binary logs on the Primary node

      • all events have reached the Replicas

      WARNING:

      The new default means that the failover time could take longer than it used to.

       

      When replicator.store.thl.stopOnDBError=true, then the Replicator will stop extracting once it is unable to update the trep_commit_seqno table in MySQL, and the Manager will perform the failover without waiting, at the risk of possible data loss due to leaving binlog events behind. All such situations are logged.

      For use cases where failover speed is more important than data accuracy, those NOT willing to wait for long failover can set replicator.store.thl.stopOnDBError=true and still use tungsten_find_orphaned to manually analyze and perform the data recovery. For more information, please see The tungsten_find_orphaned Command.

      Issues: CT-583

    • A new feature called "Cluster State Savepoints" has been implemented.

      This new functionality was created to support clean, consistent rollbacks during aborted switch and failover operations. This functionality works for both physical clusters as well as for composite clusters.

      To support this new feature, a new cluster sub-command has been added to the cctrl command - cluster topology validate, which will check and validate a cluster topology and, in the process, will report any issues that it finds. The purpose of this command is to provide a fast way to see, immediately, if there are any issues with any components of a cluster.

      Savepoints are created automatically with every switch and failover command. The savepoint is only used if there is an exception during switch or failover that is actually able to be rolled-back.

       
         

      WARNING:    

         

      Not all exceptions during switch and failover will cause a rollback.    

         

      In particular, if an exception happens during switch or failover AFTER a new primary datasource has been put online (relay or Primary) then the switch or failover operation cannot be rolled back.    

       

      The Manager is configured, by default, to hold a maximum of 50 savepoints. When that limit is hit, the Manager resets the current-savepoint-id to 0 and starts to overwrite existing savepoints, starting at 0. 

      Issues: CT-951

      For more information, see The cctrl Command.

    • Improved the ability of the manager to detect un-extracted, desirable binary log events when recovering the old Primary via cctrl after a failover.

      The cctrl recover command will now fail if:

      • any unextracted binlog events exist on the old Primary that we are trying to recover

      • the old Primary THL contains more events than the Replicas

      In this case, the cctrl recover command will display text similar to the following:

      Recovery failed because the failed Primary has unextracted events in
      the binlog. Please run the tungsten_find_orphaned script to inspect
      this events. Provided you have a recent backup available, you can
      try to restore the data source by issuing the following command:
                     datasource {hostname} restore
      Please consult the user manual at:
      https://docs.continuent.com/tungsten-clustering-6.1/operations-restore.html

      The tungsten_find_orphaned script is designed to locate orphaned MySQL binary logs that were not extracted into THL before a failover. For more information, please see The tungsten_find_orphaned Command.

      Issues: CT-996

    • Improved the ability to configure the manager's behavior upon failover.

      During a failover, the manager will now wait until the selected Replica has applied all stored THL events before promoting that node to Primary.

      This wait time can be configured via the manager.failover.thl.apply.wait.timeout=0 property.

      The default value is 0, which means "wait indefinitely until all stored THL events are applied".

      Any value other than zero invites data loss due to the fact that once the Replica is promoted to Primary, any unapplied stored events in the THL will be ignored, and therefore lost.

      Whenever a failover occurs, the Replica with most events stored in the local THL is selected so that when the events are eventually applied, the data is as close to the original Primary as possible with the least number of events missed.

      That is usually, but not always, the most up-to-date Replica, which is the one with the most events applied.

      There should be a good balance between the value for manager.failover.thl.apply.wait.timeout and the value for policy.slave.promotion.latency.threshold=900, which is the number of seconds to which a Replica must be current with the Primary in order to qualify as a candidate for failover. The default is 15 minutes (900 seconds).

      Issues: CT-1022

Bug Fixes

  • Command-line Tools

    • Installing with disable-security-controls=false or when updating using: tools/tpm update --replace-jgroups-certificate --replace-tls-certificate would generate self-signed security certs that have a 1-year expiration which will cause installs to break eventually.

      This expiration time value is controlled by the tpm command option --java-tls-key-lifetime, which is now set to 10 years or 3,650 days by default.

      Issues: CT-937

    • Updated the check_tungsten.sh command to have the executable bit set.

      Issues: CT-1037

    • Updated the check_tungsten_services and zabbix_tungsten_services commands to auto-detect active witnesses.

      Issues: CT-1043

  • Tungsten Manager

    • Fixed an issue where the ls resources command run inside of cctrl would fail to list the MANAGER entry on a Replica node.

      Issues: CT-599

    • If the pipeline source replicator goes OFFLINE, the relay will reconnect to a different Replica.

      Issues: CT-871

    • Fixed an issue where the Manager would show an exception when the MySQL check script did not get expected results.

      Issues: CT-912

    • Fixed use case where xtrabackup would timeout during backup via cctrl

      Issues: CT-1045

    • Improve ability to find needed binaries, both locally and over SSH, for commands: tungsten_find_orphaned and tungsten_is_recoverable

      Issues: CT-1053

Tungsten Clustering 6.1.1 Includes the following changes made in Tungsten Replicator 6.1.1

Release 6.1.1 contains both significant improvements as well as some needed bugfixes.

Improvements, new features and functionality

  • Core Replicator

    • Added Clickhouse applier support.

      Issues: CT-383

    • If using the dropcolumn filter during extraction, in conjunction with the Batch Applier (eg Replicating to Redshift, Hadoop, Vertica) writes would fail with a CSV mismatch error due to gaps in the THL Index.

      However, for JDBC appliers, the gaps are required to ensure the correct column mapping

      To handle the two different requirements, a new property has been added to the filter to control whether or not to leave the THL index untouched (the default) or to re-order the Index ID's

      If applying to Batch targets, then the following property should be added to your configuration. The property is not required for JDBC targets.

      --property=replicator.filter.dropcolumn.fillGaps=true

      Issues: CT-1025

Bug Fixes

  • Command-line Tools

    • Fixed an issue that would prevent reading remote binary logs when using SSL.

      Issues: CT-958

    • Fixed an issue where the command trepctl -all-services status -name watches fails.

      Issues: CT-977

    • Restored previously-removed log file symbolic links under $CONTINUENT_ROOT/service_logs/

      Issues: CT-1026

    • Fixed a bug where tpm diag would generate an empty zip file if the hostnames contain hyphens (-) or periods (.)

      Issues: CT-1032

    • Improve ability to find needed binaries for commands: tungsten_find_position, tungsten_find_seqno and tungsten_get_rtt

      Issues: CT-1054

1.31. Tungsten Clustering 6.1.0 GA (31 July 2019)

Version End of Life. 15 Aug 2024

Release 6.1.0 contains both significant improvements as well as some needed bugfixes. One of the main features of this release is MySQL 8 support.

The Tungsten Stack now supports the new MySQL 8.0 authentication plugins. Both sha256_password and caching_sha2_password (the new default) are supported by the Replicator, Manager and Connector.

More info on these authentication plugins can be found here: https://dev.mysql.com/doc/refman/8.0/en/sha256-pluggable-authentication.html

The Drizzle driver has been updated to support these new authentication methods, and the MySQL Connector/J 8 is also supported.

Behavior Changes

The following changes have been made to Tungsten Cluster and may affect existing scripts and integration tools. Any scripts or environment which make use of these tools should check and update for the new configuration:

  • Tungsten Connector

    • The Connector passThroughMode configuration option is now deprecated.

      The following passThroughMode entry will be removed from tungsten-connector/conf/connector.properties. There is currently no tpm option for this, and it is undocumented. The default will be kept to passThroughMode=true.

      # The Tungsten Connector offers an extra fast data transfer mode known as
      # pass-through. When the following switch enabled (default), the Connector
      # will directly transfer data packets between the client and the server.
      # When disabled, every native MySQL command will be translated into a JDBC call.
      passThroughMode=true

      Issues: CT-897

Known Issue

The following issues are known within this release but not considered critical, nor impact the operation of Tungsten Cluster. They will be addressed in a subsequent patch release.

  • Tungsten Connector

    • Some applications might fail to connect to the Connector with MariaDB 10+

      When using MariaDB 10+, the Connector will be confused by the 10 and will think it is a MySQL 8+ server. By default, the Connector will offer to connect with caching_sha2_password. If the application does not know how to switch authentication plugins, it will fail with a message similar to the following:

      The server requested authentication method unknown to the client [caching_sha2_password]

      As a work-around, you may specify the authentication plugin using the following tpm command option:

      --property=defaultAuthPlugin=mysql_native_password

      Issues: CT-1033

Improvements, new features and functionality

  • Command-line Tools

  • Tungsten Connector

    • The Tungsten Stack now supports the new MySQL 8.0 authentication plugins. Both sha256_password and caching_sha2_password (the new default) are supported by the Replicator, Manager and Connector.

      More info on these authentication plugins can be found here: https://dev.mysql.com/doc/refman/8.0/en/sha256-pluggable-authentication.html

      The Drizzle driver has been updated to support these new authentication methods, and the MySQL Connector/J 8 is also supported.

      In order to be fully transparent with the new defaults, when connected to a MySQL 8+ data source, the Connector will advertise caching_sha2_password as the default plugin.

      With earlier versions of MySQL (pre-8.0), the previous default mysql_native_password is used by default and advertised to the client applications.

      In order to override the default behavior, a new Connector property option for tpm, property=defaultAuthPlugin={autodetect|caching_sha2_password|mysql_native_password}, and is set to autodetect by default.

      Note that if property=defaultAuthPlugin is set to caching_sha2_password, the sha256_password authentication is automatically also supported.

      Warning

      Please note that the Connector does not support public key retrieval as of yet.

      Also note that, for backwards compatibility, the Connector forces the “CLIENT_DEPRECATE_EOF” to false, disallowing the usage of client session tracking requests (https://dev.mysql.com/doc/refman/5.7/en/session-state-tracking.html)

      Issues: CT-771

    • When logging is set to debug or trace, the Connector will print individual queries. In the past, advanced logging limited the display size of requests to 256 characters to prevent overwhelming the logs in terms of both space and filesystem I/O.

      Some customers need to display more than that, so it is now possible to adjust the size of the statements displayed in debug or trace logging modes. This is handled by a new Connector property option for tpm, property=statement.display.size.in.kb=NNN, which is defined as the maximum query length to display in Kbytes, and now defaults to 1KB.

      Warning

      Warning: setting this option to a high value while DEBUG or TRACE is enabled will quickly fill logs and disk, in addition to using up disk I/O's!

      For example, if the raw query size is 4KB, then a setting of 1KB would simply display the first 1024 bytes of the query and truncate/discard the rest from a logging perspective.

      For more information about configuring debug and trace logging, please visit Generating Advanced Diagnostic Information.

      Issues: CT-990

Bug Fixes

  • Tungsten Connector

    • OLD BEHAVIOR: If the Primary data source was not accessible when the Connector started (i.e. connection refused, etc.), the connector would still fully initialize, leading to a running Connector without an accessible data source. This has the side effects of having default configuration values for both wait_timeout and server_version instead of properly auto-detected values based on the MySQL server settings.

      NEW BEHAVIOR: The Connector will now wait indefinitely for a Primary to become available before finishing startup.

      Issues: CT-930

    • Introduced a new tpm flag allowing for tuning Connector thread stack size, which can be required in particular cases where large requests are sent as text to a connector configured for automated read/write splitting (smartscale and direct reads).

      The setting is commented out by default, leaving the JVM use its own default, generally 1024.

      Setting tpm option connector-thread-stack-size={value in kb} will override this value.

      Note

      Please note that since the new size will be allocated for each incoming connection, increasing the thread stack size will affect the total runtime memory used by the connector instance

      Issues: CT-973

Tungsten Clustering 6.1.0 Includes the following changes made in Tungsten Replicator 6.1.0

Release 6.1.0 contains both significant improvements as well as some needed bugfixes. One of the main features of this release is MySQL 8 support.

Improvements, new features and functionality

  • Command-line Tools

    • Two new utility scripts have been added to the release to help with setting the Replicator position:

      - tungsten_find_position, which assists with locating information in the THL based on the provided MySQL binary log event position and outputs a dsctl set command as output.

      - tungsten_find_seqno, which assists with locating information in the THL based on the provided sequence number and outputs a dsctl set command as output.

      Issues: CT-934

  • Core Replicator

    • A new, beta-quality command has been included called prov-sl.sh which is intended to eventually replace the current tungsten_provision_slave script.

      Currently, prov-sl.sh supports provisioning Replicas using mysqldump and xtrabackup tools, and is MySQL 8-compatible. 

      The prov-sl.sh command is written in Bash, has less dependencies compared to the current version and is meant to fix a number of issues with the current version.

      Backups are streamed from source to target so that an intermediate write to disk is not performed, resulting in faster provisioning times.

      Logs are written to $CONTINUENT_ROOT/service_logs/prov-sl.log (i.e. /opt/continuent/service_logs/prov-sl.log).

      For example, provision a Replica from [source db] using mysqldump (default):

      shell> prov-sl.sh -s {source db}

      As another example, use xtrabackup for the backup method, with 10 parallel threads (default is 4), and ssh is listening on port 2222:

      shell> prov-sl.sh -s {source db} -n xtrabackup -t 10 -p 2222

      Warning

      At the moment, prov-sl.sh does not support Composite Active/Active topologies when used with Tungsten Clustering, however it will be included in a future release.

      Issues: CT-614, CT-723, CT-809, CT-855, CT-963

    • Upgraded the Drizzle driver to support MySQL 8 authentication protocols (SHA256, caching_sha2).

      Issues: CT-914, CT-931, CT-966

    • The Redshift Applier now allows AWS authentication using IAM Roles. Previously authentication was possible via Access and Secret Key pairs only.

      Issues: CT-980

      For more information, see Redshift Preparation for Amazon Redshift Deployments.

Bug Fixes

  • Command-line Tools

    • When executing mysqldump, all Tungsten tools no longer use the --add-drop-database flag as it will prevent MySQL 8+ from restoring the dump.

      Issues: CT-935

    • Fixed a bug where tpm diag would generate an empty zip file if the hostnames contain hyphens (-) or periods (.)

      Issues: CT-1032

  • Core Replicator

    • Added support for missing charset GB18030 to correct WARN extractor.mysql.MysqlBinlog Unknown charset errors.

      Issues: CT-915, CT-932

    • Loading data into Redshift would fail with the following error if a row of data contained a specific control character (0x00 (NULL))

      Missing newline: Unexpected character 0x30 found at location nnn

      Issues: CT-984

    • Now properly extracting the Geometry datatype.

      Issues: CT-997

    • The ddl_map.json file used by the apply_schema_changes filter was missing a rule to handle ALTER TABLE statements when replicating between MySQL and Redshift

      Issues: CT-1002

    • The extract_schema_change filter wasn't escaping " (double-quotes) and the generated JSON would then cause the applier to error with

      pendingExceptionMessage: SyntaxError: missing } after property list »
      (../../tungsten-replicator//support/filters-javascript/apply_schema_changes.js#236(eval)#1)

      Issues: CT-1011

2. Tungsten Replicator Release Notes

2.1. Tungsten Replicator 7.1.1 GA (15 Dec 2023)
2.2. Tungsten Replicator 7.1.0 GA (16 Aug 2023)
2.3. Tungsten Replicator 7.0.3 GA (4 Apr 2023)
2.4. Tungsten Replicator 7.0.2 GA (19 Dec 2022)
2.5. Tungsten Replicator 7.0.1 GA (13 June 2022)
2.6. Tungsten Replicator 7.0.0 GA (29 Mar 2022)
2.7. Tungsten Replicator 6.1.24 GA (11 Dec 2023)
2.8. Tungsten Replicator 6.1.23 GA (31 Aug 2023)
2.9. Tungsten Replicator 6.1.22 GA (6 July 2023)
2.10. Tungsten Replicator 6.1.21 GA (18 Apr 2023)
2.11. Tungsten Replicator 6.1.20 GA (19 Dec 2022)
2.12. Tungsten Replicator 6.1.19 GA (17 Oct 2022)
2.13. Tungsten Replicator 6.1.18 GA (7 Sept 2022)
2.14. Tungsten Replicator 6.1.17 GA (16 May 2022)
2.15. Tungsten Replicator 6.1.16 GA (20 Dec 2021)
2.16. Tungsten Replicator 6.1.15 GA (19 Oct 2021)
2.17. Tungsten Replicator 6.1.14 GA (17 Aug 2021)
2.18. Tungsten Replicator 6.1.13 GA (27 May 2021)
2.19. Tungsten Replicator 6.1.12 GA (8 Mar 2021)
2.20. Tungsten Replicator 6.1.11 GA (21 Jan 2021)
2.21. Tungsten Replicator 6.1.10 GA (15 Dec 2020)
2.22. Tungsten Replicator 6.1.9 GA (23 Nov 2020)
2.23. Tungsten Replicator 6.1.8 GA (2 Nov 2020)
2.24. Tungsten Replicator 6.1.7 GA (5 Oct 2020)
2.25. Tungsten Replicator 6.1.6 GA (20 Aug 2020)
2.26. Tungsten Replicator 6.1.5 GA (5 Aug 2020)
2.27. Tungsten Replicator 6.1.4 GA (4 June 2020)
2.28. Tungsten Replicator 6.1.3 GA (17 February 2020)
2.29. Tungsten Replicator 6.1.2 GA (20 January 2020)
2.30. Tungsten Replicator 6.1.1 GA (28 October 2019)
2.31. Tungsten Replicator 6.1.0 GA (31 July 2019)

2.1. Tungsten Replicator 7.1.1 GA (15 Dec 2023)

Version End of Life. Not Yet Set

Release 7.1.1 contains a number of key bug fixess and improvements.

Behavior Changes

The following changes have been made to Tungsten Replicator and may affect existing scripts and integration tools. Any scripts or environment which make use of these tools should check and update for the new configuration:

  • Command-line Tools

    • The tpm diag command now collects ps -aux output in addition to ps -efl when possible.

      Issues: CT-2119

Bug Fixes

  • Installation and Deployment

    • Installations will now succeed on hosts running Ruby 3.x

      Issues: CT-2196

  • Command-line Tools

    • The tpm report command now correctly displays security info tpm options as blank when none exists.

      Issues: CT-2176

    • The tpm update command no longer fails to display foreign-owned dot-files and directories.

      Issues: CT-2190

    • The tpm mysql command now properly accepts the -e argument as a shortcut for the --execute mysql cli client argument.

      Issues: CT-2217

    • Fixes a regression in tpm cert that prevented BASE_DIR in tungsten.env from being used properly.

      Issues: CT-2237

2.2. Tungsten Replicator 7.1.0 GA (16 Aug 2023)

Version End of Life. Not Yet Set

Release 7.1.0 is the next major v7 release containing a number of important bug fixes and key new features.

Behavior Changes

The following changes have been made to Tungsten Replicator and may affect existing scripts and integration tools. Any scripts or environment which make use of these tools should check and update for the new configuration:

  • Command-line Tools

    • The tpm copy-keys command has been renamed to tpm copy and a new command has been created tpm cert copy with the same functionality.

      Issues: CT-2186

Improvements, new features and functionality

  • Behavior Changes

    • Additional logging will now be added to the replicator logs during switchover/failover operations to enable better debugging in the event of issues.

      Issues: CT-1448

  • Installation and Deployment

    • Running tpm uninstall will now save all of the Tungsten database tracking schemas for later use. There is also a new tpm keep command, which allows the tracking schemas to be saved to disk at any time in multiple formats (.json, .dmp and .cmd)

      Issues: CT-2131

    • Added tpm flag deploy-systemd as a more meaningful alias to install

      Issues: CT-2152

  • Command-line Tools

    • Added a new option --preserve-schema to the tpm uninstall command in order to leave the tracking schema in the database.

      Issues: CT-561

    • A new command tpm cert has been added to aid in the creation, rotation and management of certificates for all areas of Tungsten.

      Known limitations: Percona 5.6, and all versions of MariaDB do not provide the mysql_ssl_rsa_setup command required by tpm cert gen mysqlcerts.

      Issues: CT-2085

    • The tpm diag command now supports the --skipsudo and --nosudo arguments to prevent operations from using the sudo command. Using this option may result in tpm diag skipping/failing various gathers due to a lack of access.

      Issues: CT-2146

  • PostgreSQL Applier

    • It is now possible to configure the PostgreSQL applier in Batch mode.

      Issues: CT-2031

    • Added a new ddlscan template, ddl-mysql-postgres-staging.vm, to allow generation of staging table DDL for the PostgreSQL batch applier.

      Issues: CT-2136

  • S3 CSV Applier

    • A new applier is now available that replicates to csv files on S3 storage.

      For more information, see S3 Applier Docuemntation

      Issues: CT-606

  • Core Replicator

    • Upgraded JGroups library to 4.2.22

      Issues: CT-2011

  • Filters

    • A new shardbyrules filter has been added that will allow rule based sharding of replication based on user confgurable rules that would allow sharding at table level, whereas previoulsy sharding would only be handled at schema level.

      For more information, see shardbyrules Filter Docuemntation

      Issues: CT-2164

  • Monitoring

    • Prometheus exporters now provide the ssl cert expiration date as an epoch value in addition to the label.

      Issues: CT-2099

    • Prometheus libraries have been upgraded from version 0.8.1 to 0.16.0

      Issues: CT-2166

Bug Fixes

  • Installation and Deployment

    • Fixed RPM package script to run tpm install instead of tpm update when installing the rpm

      Issues: CT-2130

  • Command-line Tools

    • tpm uninstall would sometimes print "ERROR >> db1 >> undefined method '+' for nil:NilClass"

      Issues: CT-2104

    • The tpm command now searches more places to locate shell commands that are called, especially useful when $CONTINUENT_ROOT/share/env.sh is not sourced.

      Issues: CT-2182

  • Backup and Restore

    • tprovision will now print an error message and exit if the MySQL datadir does not exit.

      Issues: CT-1901

  • PostgreSQL Applier

    • Fixes an issue with the ddl-mysql-postgres.vm ddlscan template that caused varchar datatype lengths to be parsed incorrectly.

      Issues: CT-2019

    • Fixed a NullPointerException error occurring at startup when applying to PostgreSQL

      Issues: CT-2021

  • Core Replicator

    • Improved a query that is run by Tungsten when fetching tables metadata (column names, datatypes, etc). While it is not generally needed, the unoptimized query can run badly (especially) against old mysql versions with a lot of databases / tables. For now, the new optimized query is not used by default, but this could change in some future version.

      This can be enabled by using the following property :

      property=replicator.datasource.global.connectionSpec.usingOptimizedMetadataQuery=true

      Issues: CT-2077

    • Fixed an issue while processing geometry data with SRID 4326 that would swap longitude and latitude. This applies only to MySQL 8, as prior MySQL versions do not allow specifying the order when applying a WKB (Well-known binary) to MySQL

      Issues: CT-2172

2.3. Tungsten Replicator 7.0.3 GA (4 Apr 2023)

Version End of Life. Not Yet Set

Release 7.0.3 contains a number of key bug fixes and improvements.

Behavior Changes

The following changes have been made to Tungsten Replicator and may affect existing scripts and integration tools. Any scripts or environment which make use of these tools should check and update for the new configuration:

  • Installation and Deployment

    • enable-sudo-access is now enabled by default in Replicator only installations.

      Issues: CT-1780

  • Command-line Tools

    • The tps.pl backup tool now shows the available command-line arguments when the TAB key is pressed.

      Issues: CT-1246

    • The tungsten_skip_all command (along with alias tungsten_skip_seqno) now shows the full pendingExceptionMessage instead of just pendingError, and the More choice shows the pendingErrorEventId and the pendingError.

      Issues: CT-2030

    • The tpm command now prints a warning when running as the root OS user during operations that make changes.

      Issues: CT-2064

Improvements, new features and functionality

  • Command-line Tools

    • A new thl tail command has been added, allowing you to view the live THL changes as they are generated.

      Issues: CT-405

    • The tungsten_get_ports command now supports the /opt/continuent/tungsten/tungsten-connector/conf/interfaces.json file, if found.

      Issues: CT-2003

    • The tungsten_get_ports command now use values based on values obtained from tpm query values when possible for the greatest output accuracy.

      Issues: CT-2008

    • Shell command tab completion for Tungsten-specific commands has been updated to support recent additions and changes.

      Issues: CT-2057

  • Redshift Applier

    • Updated the Redshift JDBC driver to version 2.1.0.12

      Issues: CT-1806

  • Core Replicator

    • THL transfer protocol can now be changed dynamically. This can be done using the trepctl command tool

      shell> trepctl setdynamic -property replicator.thl.protocol.client.serialization -value DEFLATE 
      Property replicator.thl.protocol.client.serialization successfully set to DEFLATE

      This can also be done through API v2

      curl --user tungsten:secret --insecure --request POST 'https://127.0.0.1:8097/api/v2/replicator/service/alpha/thl/inflight-protocol' -d 
      '{
       "payloadType":"StringPayload",
       "payloadVersion":"1",
       "payload":
       {
       "value":"legacy"
       }
      }' 
      
      { 
       "payloadType": "StringPayload", 
       "payloadVersion": "1", 
       "payload": 
       { 
       "value": "THL inflight protocol is now set to legacy" 
       } 
      }

      Issues: CT-1862

Bug Fixes

  • Command-line Tools

    • The dsctl command now works correctly against MongoDB targets.

      Issues: CT-580

    • Fixed rsync availability check in TPM to error out early when the executable is not found in the path

      Issues: CT-1028

    • The trepctl reset command now works correctly against MongoDB targets.

      Issues: CT-1233

    • The tpm ask keys command now lists all available keys properly.

      Issues: CT-2041

    • The tpm update command no longer aborts with a Dataservice was NOT removed error when service names with a hyphen are defined in the configuration. We strongly discourage the use of hyphens (-) in service names and recommend underscores (_) instead.

      Issues: CT-2052

    • The tpm validate-update command now performs the delete-service checks properly.

      Issues: CT-2054

    • The tungsten_find_events command no longer fails with error Undefined subroutine &main::info

      Issues: CT-2058

    • The tpm diag command will now locate and kill any remaining tpm diag child processes on the calling host.

      Issues: CT-2068

    • Suppressed wrong tpm errors with autodetect hosts.

      Issues: CT-2072

  • PostgreSQL Applier

    • Improved some data type mappings for MySQL to PostgreSQL replication.

      Issues: CT-2069

  • Core Replicator

    • Fixed an issue that was occurring when an ALTER USER is applied on a mysql instance that uses lower case identifiers.

      Issues: CT-2024

    • Fixed an issue where an extracted JSON field could be invalid, with doubled commas in JSON arrays.

      Issues: CT-2049

    • Fixed an issue where status would fail if issued while a service was reconfiguring (in the middle of a setrole command for example).

      Issues: CT-2081

2.4. Tungsten Replicator 7.0.2 GA (19 Dec 2022)

Version End of Life. Not Yet Set

Release 7.0.2 contains a number of key bug fixes and improvements.

Behavior Changes

The following changes have been made to Tungsten Replicator and may affect existing scripts and integration tools. Any scripts or environment which make use of these tools should check and update for the new configuration:

  • Command-line Tools

    • The various user-xxxx.log files are no longer generated

      Issues: CT-1914

    • The check_tungsten.sh was deprecated in release 6.1.18 and has now been removed from this release.

      Issues: CT-1939

    • Both the tungsten_get_ports and tpm report commands have been updated to use the ss OS command when the netstat OS command is unavailable or deprecated.

      Issues: CT-2007

  • Core Replicator

    • repl_svc_extractor_multi_frag_service_detection is now turned ON by default. Event shards are determined at extraction time. With fragmented events, the shard cannot be determined by only reading the first fragment, but needs to check the last fragment as well. With this setting turned OFF, there is no issue with pipelines that don't need it, i.e. no parallel apply downstream replicas. However, as this is done at extract time, THL contains this information, and adding or changing a replica using parallel apply could introduce issues.

      Note

      It can be disabled if you see a performance overhead but this should be done with caution. For Aurora<>Aurora Active/Active deployments it is essential that this property be left ON.

      Issues: CT-1959

Improvements, new features and functionality

  • Command-line Tools

    • A new -c option is now available with some trepctl commands that can be used in conjunction with the -r option to indicate the number of times to refresh before automatically terminating. For example, the following command:

      shell> trepctl perf -r 3 -c 10

      Will refresh the output every 3 seconds, 10 times.

      Issues: CT-679

    • The trepctl status will now show the last known applied seqno and latency.

      This information is stored on disk at regular intervals (10s minimum) so as not to overload the replicator, therefore the value can be shown as slightly old dependant on when the status command was issued.

      By default, this feature is disabled. It can be enabled by setting the following parameter in the configuration :

      svc-applier-last-applied-write-interval=20

      This will write current position to disk every 20 seconds. This information is also exported by the Prometheus exporter.

      If the service is online, it will display the current value (the same as appliedLastSeqno and appliedLatency)

      shell> trepctl status 
      Processing status command... 
      NAME VALUE 
      ---- ----- 
      appliedLastEventId : mysql-bin.000017:0000000151329854;70 
      appliedLastSeqno : 999 
      appliedLatency : 347707.0 
      ... 
      lastKnownAppliedLatency: 347707.0 
      lastKnownAppliedSeqno : 999 
      ...

      Issues: CT-1823

    • The tmonitor command now accepts cli args to specify the ports and will auto-configure the ports if they have been changed via the Tungsten configuration.

      Issues: CT-1919

    • The tpm command calls to glob have been improved to be more strict and compliant.

      Issues: CT-1940

    • The tpm ask stages and tpm ask allstages commands have been added to display the Replicator stages for the current node (stages) and the stages for each role (allstages).

      Issues: CT-1943

    • The tpm ask command has five new variables available: dsrole & dsstate for the current datasource, and trrole & trstate for the current replicator, and nodeinfo which displays all 4 of the new variables.

      Issues: CT-1944

    • The tpm report command now prints the hostname and listener ports where available when using the --extra|-x option or the new --ports option.

      Issues: CT-1969

    • A new dsctl option has been added to the thl command, and a new -event option added to thl list

      Full details and examples of these new options can be found here

      Issues: CT-2012

  • Core Replicator

    • Per-service tuning of the replicator thl directory is now possible for multi-service replicator-only installs as well as for clustering. The given value should be the base directory, to which tungsten will add the service name. For example, the following entry in the tungsten.ini:

      [alpha]
      ...
      ...
      thl-directory=/drv1/thl
      ...

      Would result in the THL being placed in /drv1/thl/alpha

      Note

      Update of thl directory is only available when tpm is called from the staging installation directory, NOT from the running directory.

      Issues: CT-1927

    • A new replicator role (thl-applier) has been added to allow a replicator service to apply its locally available THL, without pulling from a remote host

      Issues: CT-1936

    • Added a way to configure the maximum number of rows that can be grouped together when applying row based events for multiple insert or delete statements.

      For these properties to be in effect, you must ensure that optimize-row-events=true is either explicitly set in your configuration, or not present (since it will be enabled by default)

      For example, the following settings will limit the number of inserted or deleted rows applied at once to 10:

      optimize-row-events-limit-insert-rows=10 
      optimize-row-events-limit-delete-rows=10

      The default values if not specified will be 50 for inserts and 100 for deletes. Note that for deletes to be optimized, the affected table MUST have a single column PK.

      Issues: CT-1980

  • API

    • A new log file has been added for the REST API, it is as follows:

      • service_logs/replicator-api.log

      Issues: CT-1983

Bug Fixes

  • Installation and Deployment

    • ddlscan, dsctl and tungsten_send_diag are now added to the aliases.sh script.

      Issues: CT-813

    • Fixes issues where fixed properties and filters passed to tpm in service stanzas were not being configured correctly

      Issues: CT-1463

    • No longer using Tanuki wrapper functionality to print jvm version, which was creating defunct java processes at startup, now using internal code.

      Issues: CT-1876

  • Command-line Tools

    • Running tpm update on a standalone Replicator deployment no longer prints an uninitialized value error.

      Issues: CT-1807

    • Fixes an issue that prevented ddlscan from connecting to MySQL if SSL was enabled.

      Issues: CT-1808

    • The tungsten_skip_seqno command no longer fails when -i is specified, and now properly filters using --filter when there is a long error message.

      Issues: CT-1877

    • The tpm command now allows any case for section entries (i.e. [alpha_FROM_beta]) in the INI files.

      Issues: CT-1879

    • The tpm diag command now passes when the nodename defined in the tungsten.ini is the shortname, and DNS returns the FQDN.

      Issues: CT-1908

    • The tpm diag command now gathers the mysql.log file when SSL is enabled in the server.

      Issues: CT-1920

    • Note

      This fix was released in Tungsten Clustering and Tungsten Replicator 7.0.2 Build 161.

      The tpm command checks for the existence of the mysql command-line client when installing/upgrading. The process will no longer abort with an error on non-MySQL targets such as heterogeneous replicator appliers, or Active-Witness hosts.

      Issues: CT-1924, CT-2018

    • Fixes an issue that prevented dsctl from connecting to MySQL if SSL was enabled.

      Issues: CT-1928

    • The tpm mysql command will now gracefully handle being run on a non-database node.

      Issues: CT-1946

    • The tpm mysql command no longer aborts with an access denied error on CentOS 6.

      Issues: CT-1977

    • Both TungstenAPI and tpasswd now properly update .passwords.store.orig backup file so that proper manipulation of passwords won't trigger tpm update failure

      Issues: CT-1981

  • Core Replicator

    • Fixed a possible issue when recovering an old primary as a replica after failover when parallel apply is enabled, that could lead the replica to be unable to come online and require a reprovisioning of this replica.

      Issues: CT-1890

    • Fixes an issue that prevented geometry datatypes with SRID from being replicated.

      Issues: CT-1904

    • Fixed an issue where the replicator would hang after applying a DROP TABLE event, that originally failed on the primary, but got logged into the binlog.

      Issues: CT-1973

    • Fixed a parsing issue that would prevent the replicator from correctly detecting a CREATE TABLE statement with START TRANSACTION

      Issues: CT-1987

    • Fixed an issue in the applier to ensure it is not committing too often with a multi-service replicator (active/active for example, more specifically with AWS Aurora, or an unprivileged MySQL, backend).

      Issues: CT-2004

  • Filters

    • Fixed an issue where the dropsqlmodes filter would fail to remove invalid sql modes from a multi-statement event

      Issues: CT-1993

2.5. Tungsten Replicator 7.0.1 GA (13 June 2022)

Version End of Life. Not Yet Set

Release 7.0.1 is the first minor release for the v7 release. This release contains a number of new features, minor bug fixes and a few key behavior changes, but importantly contains a critical bug fix for environments that use mixed-case table and database names, that could cause replication failures.

Customers already running v7.0.0 in production enviornments should upgrade to this release as soon as possible

Behavior Changes

The following changes have been made to Tungsten Replicator and may affect existing scripts and integration tools. Any scripts or environment which make use of these tools should check and update for the new configuration:

  • Installation and Deployment

    • A better error message is now provided when keystore password and enclosed private key password don't match.

      Issues: CT-1853

  • Command-line Tools

    • The tpm ask summary command is now sorted by key when the Perl Tie::IxHash module is installed.

      Issues: CT-1820

    • The tpm diag command now gathers the output of the tpm ask summary command

      Issues: CT-1827

    • The tmonitor command now calls sudo internally as needed, removing the need for the tmonitor command to be run with sudo for the install and remove sub-commands.

      Issues: CT-1846

  • Core Replicator

    • The default value for the tpm property repl-svc-fail-on-zero-row-update has been changed from warn to stop

      Warning

      This is a significant change in behavior and could cause replicators, specifically cross-site replicators, in an Active/Active environment to stop and error more frequently.

      To maintain original behavior you will need to add this property with the value of warn to your configuration, providing you fully understand the inherent risk of ignoring such warnings.

      Issues: CT-1856

Known Issues

The following issues are known within this release but not considered critical, nor impact the operation of Tungsten Replicator. They will be addressed in a subsequent patch release.

  • Installation and Deployment

    • After starting up Tungsten components, a defunct process for each running component can be found in the process listing.

      Whilst this does not cause any issues, it could generate unnecessary alerts for customers monitoring.

      The cause has been identified and affects version 7.0.0 and 7.0.1. This will be fixed in the next 7.0.2 release.

      Issues: CT-1876

Improvements, new features and functionality

  • Command-line Tools

    • Added the ability to turn auto recovery on or off dynamically, removing the need to run tpm update.

      This is done by running the following command:

      shell> trepctl -service servicename setdynamic -property replicator.autoRecoveryMaxAttempts -value <number>

      Note

      The service must be offline before changing the property

      Issues: CT-1088

    • A new tpm report sub-command has been added. By default, tpm report will generate a security report for all available communication channels on a per-node basis. The purpose of tpm report is to provide easy access to all of the settings that pertain to a specific topic. The default (and only) topic at this time is the security stance. More topics will be added over time.

      Issues: CT-1793, CT-1857

    • The tapi command now supports the creation and modification of multiple APIv2 admin users. Two new cli args have been added: --create-user and --create-pass.

      Issues: CT-1801

    • The tapi command now auto-detects the protocol, either http or https

      Issues: CT-1803

    • Added JMX encryption and authentication status to trepctl -verbose and connector command outputs

      Issues: CT-1813

    • The tapi command will now output service names with the new option --includeServiceName, or if there is more than one service name found..

      Issues: CT-1860

  • Core Replicator

    • The thl list command now displays an approximative field size in bytes for row-based replication.

      Issues: CT-1847

  • Security

    • Replaced official log4j library with a secured version in which all vulnerable classes have been removed. This prevents exposing the software following a (user) misconfiguration of the log4j properties

      Issues: CT-1810

  • API

    • Added a way to retrieve service properties through the API, for example, to check a specific service parameter, run the following API call :

      shell> curl https://127.0.0.1:8097/api/v2/replicator/service/alpha/properties?filter=replicator.global.db.sslEnabled
      { 
       "payloadType": "PropertiesListPayload", 
       "payloadVersion": "1", 
       "payload": { 
       "replicator.global.db.sslEnabled": "false" 
       } 
      }

      Issues: CT-1802

    • Added new top level REST API call to display RMI SSL status, run the following API call :

      shell> curl https://127.0.0.1:8097/api/v2/rmiSecurity

      Issues: CT-1812

Bug Fixes

  • Installation and Deployment

    • The tpm update command now properly handles v7 -> v7 upgrades.

      Issues: CT-1815

    • deployall script now properly displays the executable prefix for restarting services, typically mm_treplicator in Multi-Site/Active-Active replicator-only installs, as well as the full path to component binaries

      Issues: CT-1835

    • tpm would fail on keystores containing special characters in alias names.

      Issues: CT-1852

  • Command-line Tools

    • Worked around a ruby issue where DNS name resolution was sometimes preferred to file-based resolution in TPM. The following logic now applies: first trying file-based host name resolution, upon failure try DNS, then through ping command

      Issues: CT-1653

    • Improves the tpm diag command so that it waits 2 seconds for replicator thread dump to complete.

      Issues: CT-1792

    • Fixed an `Undefined subroutine` call error in tapi --create

      Issues: CT-1798

    • tpm now properly reports errors upon timeout executing commands. Typically, when a host is down, when upgrading or installing, tpm will now properly report that pinging this host failed.

      Issues: CT-1819

    • TPM would fail to validate data sources when using ssl-capath [directory] rather than ssl-ca [file] in my.cnf.

      This fixes only the validation part, automated truststore creation by TPM will not be possible at this point, a full tungsten-truststore.ts will have to be passed through --java-truststore-path

      Issues: CT-1826

    • The tapi command now auto-switches protocol (http or https) properly.

      Issues: CT-1829

    • replicator start offline would not properly pass the offline argument to systemd scripts when installed through deployall.

      Issues: CT-1836

    • undeployall script would not properly uninstall systemd services on some distributions including amazon linux.

      Issues: CT-1845

  • Redshift Applier

    • If the optional RedshiftCopyOptions option was ommitted from the s3-config.json file, the replicator would fail to start with an undefined error.

      Issues: CT-1814

  • Core Replicator

    • Due to a change in binlogging behavior from MySQL 5.7 onwards (Including Aurora) when using the standalone Tungsten Replicator to configure bi-directional replication with unprivileged access (ie unable to control binlogging behavior), certain DDL statements failed to be tagged correctly causing them to be re-applied to the originating source database, resulting in replicator errors.

      Issues: CT-1083

    • Fixed the EnumToString and pkey filters to renew their database connections (every hour by default). This can be changed with the following properties:

      property=replicator.filter.enumtostring.reconnectTimeout=3600
      property=replicator.filter.pkey.reconnectTimeout=3600

      Issues: CT-1786

    • tpm update would fail with message "The host is configured to be '' but it is currently operating as a 'master'" when a dynamic replicator configuration file is found

      Issues: CT-1848

    • Fixes issues with PrimaryKeyFilter:

      • It could fail from correctly fetching primary key columns for tables with mixed upper and lower case letters in table names

      • Alter user statements could make it fail

      Issues: CT-1861

    • Fixed a NullPointerException that could happen if a replicator was put offline during startup. This had mostly no consequence except the exception being logged.

      Issues: CT-1863

2.6. Tungsten Replicator 7.0.0 GA (29 Mar 2022)

Version End of Life. Not Yet Set

Release 7.0.0 is a major release introducing many new features including a fully documented API. There are a large number of bug fixes and improvements in all areas of the product, and a number of key behavior changes, most significantly being Security is now enabled by default.

Behavior Changes

The following changes have been made to Tungsten Replicator and may affect existing scripts and integration tools. Any scripts or environment which make use of these tools should check and update for the new configuration:

  • Installation and Deployment

    • An RPM update now issues --replace-release

      Issues: CT-1708

  • Command-line Tools

    • The tpm diag command now uses tar czf instead of the zip command to compress the gathered files. The zip command is no longer a pre-requisite for tpm diag.

      Issues: CT-1253

    • tungsten_set_position has been deprecated and no longer available in this release. dsctl should be used instead.

      tungsten_provision_slave has now been renamed to tprovision

      Issues: CT-1302

    • The tungsten_find_orphaned script now creates a log file every time it runs which is stored in the configured temporary directory (/tmp/ by default, tpm query values temp_directory). This is to allow for easier troubleshooting and visibility during automatic execution.

      Issues: CT-1447

    • tpm now accepts chrony as a valid time synchronization software

      Issues: CT-1462

    • The tpm diag command now uses the ss (socket status) command in place of netstat on SUSE and other operating systems that have deprecated netstat.

      Issues: CT-1483

    • The tpm diag command now gathers the /etc/os-release file when located. Also now using the ip command on systems where ifconfig and/or route is deprecated.

      Issues: CT-1496

    • Changed output of thl purge command when no lower and upper bounds are given from 'Deleting events where' to 'Deleting all events'.

      Issues: CT-1738

  • Backup and Restore

    • tungsten_provision_slave has now been renamed to tprovision.

      Issues: CT-1436

  • Core Replicator

    • Replicator will now check after extracting a mysql STOP_EVENT whether a new binlog file was created, actually handling this event as a ROTATE EVENT.

      This decreases the time that would be needed after STOP_EVENT to extract a new event..

      Issues: CT-1349

  • Other Issues

    • Log files for each component now have the same date and time stamp format.

      Issues: CT-1669

  • API

    • API calls triggering configuration changes are protected by a flag, i-am-sure=true, in order to avoid unwanted, potentially dramatic, configuration changes. This applies to:

      • configuration/module/servicesmap

      • reset

      • offline

      • online

      • onhold

      • addDataService

      • addDataSource

      Issues: CT-1317

Known Issues

The following issues are known within this release but not considered critical, nor impact the operation of Tungsten Replicator. They will be addressed in a subsequent patch release.

  • Installation and Deployment

    • After starting up Tungsten components, a defunct process for each running component can be found in the process listing.

      Whilst this does not cause any issues, it could generate unnecessary alerts for customers monitoring.

      The cause has been identified and affects version 7.0.0 and 7.0.1. This will be fixed in the next 7.0.2 release.

      Issues: CT-1876

  • Command-line Tools

    • The check_tungsten_online command returns a Replicator offline error on active witness hosts.

      Issues: CT-1783

Improvements, new features and functionality

  • Installation and Deployment

    • Two new tpm options have been included as part of the new API in this release.

      The options are used for setting the API admin user credentials and are as follows:

      Issues: CT-1327

    • Support now included for MariaDB 10.3+

      Issues: CT-1276, CT-1433

    • Support has been added for Java 17 LTS

      Issues: CT-1706

  • Command-line Tools

    • A new tpm option delete-service is now available to simplify the removal of replicator services.

      Issues: CT-327

    • Prometheus exporters mysqld_exporter and node_exporter are now included with the distribution packages.

      A new command line tool tmonitor is now available for the management and testing of external Prometheus exporters (node and mysqld), and for the testing of internal exporters (Replicator).

      Issues: CT-960

    • A new tpm option purge-thl and a new script tungsten_purge_thl have been added to allow easier and more intelligent THL purging across all nodes in a topology.

      This allow you to purge THL files based on the following rules:

      • Gather the last applied seqno from all Replica nodes and take the lowest one

      • Find the current THL file which contains that seqno, then locate the previous one

      • Construct a thl purge command to remove thl thru the last seqno in the prev file

      The default behavior is to display the needed commands for the admin to execute manually.

      Issues: CT-1273

    • tpm diag now collects routing table information via route -n , and has two new command-line arguments: --include and --groups.

      --include specifies a comma-separated list of subroutines to include. Any gather subroutine not listed will be skipped.

      --groups specifies a comma-separated list of subroutine groups to include. Any group not listed will be skipped.

      Issues: CT-1322

    • tungsten_send_diag now supports a new command-line argument, --cleanup, which will cause the removal of the diagnostic archive file generated using the --diag argument.

      Issues: CT-1360

    • With the release of APIv2, a new cli tool has been introduced to allow easier access called tapi.

      In addition, the vast majority of Tungsten cli tools have been updated to optionally use the APIv2 interface when desired.

      The Nagios and Zabbix checks are also available via APIv2 using the tapi tool.

      Issues: CT-1454

    • The tungsten_purge_thl command is now a wrapper for the tpm purge-thl command.

      Issues: CT-1488

    • The tmonitor command now has better help text and more options to ease usage, including --filter to allow easy viewing of the tmonitor test output.

      Issues: CT-1585

    • A new option to print the merged logs to STDOUT has been added to tungsten_merge_logs (--stdout|-O).

      The tpm command suite now properly supports the --profile argument to specify a Tungsten json configuration file in place of the installed tungsten.cfg.

      Issues: CT-1680

    • A new command (error) has been added to trepctl to output a full stack trace of the last error, if any.

      shell> trepctl -service <serviceName> error
      
      Event application failed: seqno=10 fragno=0 message=Table hr.regions not found in database. Unable to generate a valid statement.
      com.continuent.tungsten.replicator.applier.ApplierException: Table hr.regions not found in database. Unable to generate a valid statement.
      at com.continuent.tungsten.replicator.applier.JdbcApplier.getTableMetadata(JdbcApplier.java:582)
      at com.continuent.tungsten.replicator.applier.JdbcApplier.fillColumnNames(JdbcApplier.java:494)
      at com.continuent.tungsten.replicator.applier.JdbcApplier.getColumnInformation(JdbcApplier.java:1236)
      at com.continuent.tungsten.replicator.applier.MySQLApplier.applyOneRowChangePrepared(MySQLApplier.java:418)
      at com.continuent.tungsten.replicator.applier.JdbcApplier.applyRowChangeData(JdbcApplier.java:1460)
      at com.continuent.tungsten.replicator.applier.JdbcApplier.apply(JdbcApplier.java:1576)
      at com.continuent.tungsten.replicator.applier.ApplierWrapper.apply(ApplierWrapper.java:100)
      at com.continuent.tungsten.replicator.pipeline.SingleThreadStageTask.apply(SingleThreadStageTask.java:871)
      at com.continuent.tungsten.replicator.pipeline.SingleThreadStageTask.runTask(SingleThreadStageTask.java:601)
      at com.continuent.tungsten.replicator.pipeline.SingleThreadStageTask.run(SingleThreadStageTask.java:185)
      at java.base/java.lang.Thread.run(Thread.java:834)

      Issues: CT-1747

  • Redshift Applier

    • Adds a new option redshiftCopyOptions that can be supplied in the s3.json file to include valid options that can be included in the Redshift COPY command when loading csv data from S3 into Redshift staging tables. See example below.

      {
       "awsS3Path" : "s3://your-bucket-for-redshift/redshift-test",
       "awsIAMRole" : "arn:iam-role",
       "redshiftCopyOptions" : "ACCEPTANYDATE ACCEPTINVCHARS TRUNCATECOLUMNS",
       "cleanUpS3Files" : "true"
      }

      Issues: CT-1298

  • PostgreSQL Applier

    • Templates have been included for ddlscan to enable MySQL to Postgres Schema translation.

      Issues: CT-1736

  • Core Replicator

    • It is now possible to compress and/or encrypt THL on disk. For more information on using these features see https://docs.continuent.com/tungsten-replicator-7.0/thl-compress-encrypt.html

      Issues: CT-630

    • The replicator will now be able to handle new SQL_MODES available in later releases of MySQL and MariaDB, these are as follows:

      • MySQL: TIME_TRUNCATE_FRACTIONAL

      • MariaDB: TIME_ROUND_FRACTIONAL, SIMULTANEOUS_ASSIGNMENT

      Issues: CT-1362

    • In-Flight THL Compression is now available.

      For full details on enabling this feature, refer to this page

      Issues: CT-1420

    • Added a new property to allow configuration of character set used during batch loading into MySQL targets.

      Default is ut8mb4. Override using the following:

      property=replicator.applier.dbms.loadCharset=utf8

      Issues: CT-1647

  • Security

    • Tungsten can now install on default CentOS and RedHat 8 with tightened security settings thanks to support of TLSv1.3

      Issues: CT-1359

    • Tungsten software now undergoes a rigorous security scan during QA. We also check included open-source/3rd-party software.

      Issues: CT-1579

  • Monitoring

    • A number of new metrics have been added to the Prometheus exporters.

      Issues: CT-1266, CT-1615

  • Platform Specific Deployments

    • ARM 64 bit processor support added (linux aarch64)

      Note

      Note that at time of release, there is currently no xtrabackup binary available for ARM.

      Issues: CT-1619, CT-1620

  • Other Issues

    • General improvements in MySQL 8.0 support.

      Issues: CT-1346

    • IPv6 host addresses are now fully supported.

      Can be enabled with the following configuration property:

      prefer-ip-stack="6"

      By default, IPv4 is enabled, which equates to the value of "4" in the above property.

      Issues: CT-1537

Bug Fixes

  • Installation and Deployment

    • Default systemd configuration files for Tungsten components no longer specifies the tungsten group to execute the command. This will prevent file access issues when tungsten user belongs to several groups.

      Issues: CT-1550

    • When services are deployed with systemd and MySQL could not start due to an error, tpm would not be able to later start MySQL

      Issues: CT-1734

  • Command-line Tools

    • Issuing dsctl set [--reset] against Aurora source and targets would fail with Access Denied errors. This is due to the inability to disable Binary Logging for specific transactions which dsctl issues. The errors did not prevent dsctl from working, however they were misleading.

      Issues: CT-1040

    • In certain cases, tprovision would not be able to find the binary log position of the backup when taken from a primary. This has been fixed.

      Issues: CT-1085

    • Fixes a bug in tprovision when using xtrabackup version 8, due to changes in xtrabackup binaries.

      Issues: CT-1248

    • The tpm connector command now handles special characters in the password string.

      Issues: CT-1258

    • The tpm update command will now exit with an error if any files not owned by the configured Tungsten OS user are found in the Tungsten installation directory.

      For example, if the OS user is tungsten and the installation directory is /opt/continuent, containing the file /opt/continuent/thl/archived_thl.zip owned by root would cause something like the following error to be produced, and tpm update would exit:

      Foreign-owned files found!
      
      Located files in the Tungsten installed directory /opt/continuent
      that are not owned by the Tungsten OS user (tungsten):
      
      /opt/continuent/thl/archived_thl.zip
      
      Please change the ownership of these files to OS user "tungsten"
      using the chown command as root via sudo, then rerun the `tpm update` command.
      
      For example:
      shell> sudo chown -R tungsten:tungsten /opt/continuent

      Issues: CT-1260

    • Fixes a security issue within the tpm diag command.

      Issues: CT-1295

    • tungsten_send_diag no longer prints an error about Use of uninitialized value $diagArgs in concatenation.

      Issues: CT-1354

    • The tpm command no longer prints an error when run with no other command-line arguments.

      Issues: CT-1373

    • The tpm command no longer aborts with a Use of uninitialized value error when a stray tungsten.cfg file exists under $CONTINUENT_ROOT

      Issues: CT-1394

    • Fixes a monitoring bug with users using caching_sha2_password.

      Issues: CT-1406

    • The tpm command now checks the local host’s shortname in addition to the FQDN returned by the hostname command upon initialization.

      This fixes a bug that prevented tpm mysql from working on applier hosts running MySQL.

      Issues: CT-1414

    • tprovision (formerly tungsten_provision_slave) may fail to provision if the MySQL data directory was not accessible to the tungsten user.

      Issues: CT-1475

    • The tmonitor command no longer fails on Debian 9 and Ubuntu.

      Issues: CT-1485

    • All tpm sub-commands now handle command-line arguments more intelligently.

      Issues: CT-1487

    • The tpm purge-thl command now handles command-line arguments more intelligently.

      Issues: CT-1489

    • The tpm diag command now properly collects the system information file on Debian systems.

      Issues: CT-1492

    • Database monitoring logs are now reporting the correct error number and SQL state when database errors occur.

      Issues: CT-1497

    • The tpm update command now handles updates/upgrades more gracefully when the previous version did not have the latest tpm framework..

      Issues: CT-1506

    • The tpm ask command no longer calls Data::Dumper when it is not available.

      Issues: CT-1626

    • tpm now parses the MYSQL SSL related setting correctly.

      Issues: CT-1662

    • Fixes an issue where the deployall command would create a root owned wrapper.log in the ./tools directory.

      Issues: CT-1664

    • When MySql services where badly installed, some distribution could show a “not-found” status within systemctl, confusing tpm

      Issues: CT-1677

    • The tpm command now communicates properly when there is no INI configuration file or staging-method deploy.cfg configuration defined.

      Issues: CT-1712

    • The tpm diag command now handles Multi-Site/Active-Active topologies better.

      Issues: CT-1718

    • tungsten_monitor.rb script no longer uses sudo to send emails if the configuration doesn't allow it.

      Issues: CT-1737

    • The tpm diag command now handles zero-length mysqld.log files gracefully.

      Issues: CT-1740

  • Backup and Restore

    • Fixed an issue where an xtrabackup generated by the replicator would fail to be restored using trepctl restore command.

      Issues: CT-1575

  • Oracle Applier

    • Fixes an issue causing an error when using the ddl-mysql-oracle.vm template with ddlscan to convert MySQL schema DDL into Oracle DDL

      Issues: CT-1326

    • Fixes a bug in the MySQL to Oracle ddlscan templates that prevented them from executing.

      Issues: CT-1735, CT-1756

  • Heterogeneous Replication

    • Fixes an issue where a heterogeneous (non-MySQL) applier would try to run MySQL syntax to set the timezone in use.

      Issues: CT-1324

  • Core Replicator

    • The replicator metadata cache will now correctly handle table names when lower_case_table_names=1 is set in the MySQL configuration.

      Issues: CT-651

    • When using parallel apply, the replicator would error with a Foreign Key constraint error if statements were issued against two or more objects that shared the same name, but with different case sensitivity, for example:

      mysql> create table testtable;
      mysql> drop table testtable;
      mysql> create table TestTable;

      Issues: CT-1259

    • A change in the way MySQL logs CREATE TABLE AS SELECT in the Binary Logs from v8.0.20 onwards, meant these transactions would previously fail.

      Warning

      Whilst these statements will now replicate, it mut be noted that in the event of a failure during the data load, the initial CREATE statement won't be rolled back, and therefore care must be taken when using this type of DDL.

      Note

      This only affects customers using MySQL v8.0.20+ running with ROW based replication. An alternative workaround to ensure correct rollback on failure, would be to run the statement with STATEMENT based replication for the session. This will also provide better performance for larger tables.

      Issues: CT-1301

    • Fixes occurences of NullPointerException that would occur when bringing the replicator online before MySQL was started.

      Issues: CT-1348

    • For row based events, SQL modes were not displayed in the THL output. This is now fixed.

      Issues: CT-1440

    • When connecting to a THL server, a client will now connect to the next available host in its THL uri, if the first does not have the sequence number that the client requires. The client will then fail only if none of the hosts from the uri can provide the needed sequence number.

      Issues: CT-1558

    • Fixed an issue when using Parallel apply that would show a NullPointerException in case an event could either not be found or be corrupted in THL. This will now display a correct message Missing or corrupted event from storage

      Issues: CT-1722

    • Fixed an issue with the batch loader where DATETIME and TIME datatypes with milliseconds would be replicated as .000 instead of the real millisecond value.

      Issues: CT-1739

    • Fixed an issue where trepctl was leaving JMX connections opened.

      Issues: CT-1752

    • Added more debug information for detecting possible hanging connections while a THL client connects to the THL server. Also, added socket timeout for the connection initialization

      Issues: CT-1760

  • Filters

    • Includes previously missing template file to enable easy configuration of the dbrename filter.

      Issues: CT-1350

    • The BidiRemoteSlaveFilter could fail to correctly flag fragmented events in unprivileged environments (Aurora, for example) In such an environment (multi-active, unprivileged database access), a new setting was introduced to force extraction process to read ahead to the last fragment to detect the service name (false by default). Enabled with repl_svc_extractor_multi_frag_service_detection=true

      Issues: CT-1351

  • Security

    • log4j libraries updated to v2.17.1 specifically to mitigate risk of exposure to the 0-day vulnerbaility detected in log4j v2.14

      Issues: CT-1703

2.7. Tungsten Replicator 6.1.24 GA (11 Dec 2023)

Version End of Life. 15 Aug 2024

Release 6.1.24 was released to address a bug in the Tungsten Clustering product and there are no specific changes in Tungsten Replicator.

2.8. Tungsten Replicator 6.1.23 GA (31 Aug 2023)

Version End of Life. 15 Aug 2024

Release 6.1.23 was released to address a minor bug in the Tungsten Clustering product and there are not specific changes in Tungsten Replicator.

2.9. Tungsten Replicator 6.1.22 GA (6 July 2023)

Version End of Life. 15 Aug 2024

Release 6.1.22 contains a number of minor bug fixes.

Bug Fixes

  • Installation and Deployment

    • Fixed RPM package script to run tpm install instead of tpm update when installing the rpm

      Issues: CT-2130

  • PostgreSQL Applier

    • Fixed a NullPointerException error occurring at startup when applying to PostgreSQL

      Issues: CT-2021

  • Core Replicator

    • A change in the way MySQL logs CREATE TABLE AS SELECT in the Binary Logs from v8.0.20 onwards, meant these transactions would previously fail.

      Warning

      Whilst these statements will now replicate, it mut be noted that in the event of a failure during the data load, the initial CREATE statement won't be rolled back, and therefore care must be taken when using this type of DDL.

      Note

      This only affects customers using MySQL v8.0.20+ running with ROW based replication. An alternative workaround to ensure correct rollback on failure, would be to run the statement with STATEMENT based replication for the session. This will also provide better performance for larger tables.

      Issues: CT-1301

    • Fixed an issue where an extracted JSON field could be invalid, with doubled commas in JSON arrays.

      Issues: CT-2049

    • Improved a query that is run by Tungsten when fetching tables metadata (column names, datatypes, etc). While it is not generally needed, the unoptimized query can run badly (especially) against old mysql versions with a lot of databases / tables. For now, the new optimized query is not used by default, but this could change in some future version.

      This can be enabled by using the following property :

      property=replicator.datasource.global.connectionSpec.usingOptimizedMetadataQuery=true

      Issues: CT-2077

2.10. Tungsten Replicator 6.1.21 GA (18 Apr 2023)

Version End of Life. 15 Aug 2024

Release 6.1.21 contains a number of critical bug fixes and improvements.

Behavior Changes

The following changes have been made to Tungsten Replicator and may affect existing scripts and integration tools. Any scripts or environment which make use of these tools should check and update for the new configuration:

  • Command-line Tools

    • The tungsten_skip_all command (along with alias tungsten_skip_seqno) now shows the full pendingExceptionMessage instead of just pendingError, and the More choice shows the pendingErrorEventId and the pendingError.

      Issues: CT-2030

Improvements, new features and functionality

  • Platform Specific Deployments

    • ARM 64 bit processor support added (linux aarch64)

      Note

      Note that at time of release, there is currently no xtrabackup binary available for ARM.

      Issues: CT-1619, CT-1620

Bug Fixes

  • Command-line Tools

    • The tpm ask keys command now lists all available keys properly.

      Issues: CT-2041

2.11. Tungsten Replicator 6.1.20 GA (19 Dec 2022)

Version End of Life. 15 Aug 2024

Release 6.1.20 contains a number of critical bug fixes and improvements.

Behavior Changes

The following changes have been made to Tungsten Replicator and may affect existing scripts and integration tools. Any scripts or environment which make use of these tools should check and update for the new configuration:

  • Command-line Tools

    • Both the tungsten_get_ports and tpm report commands have been updated to use the ss OS command when the netstat OS command is unavailable or deprecated.

      Issues: CT-2007

Improvements, new features and functionality

  • Command-line Tools

    • A new command has been added called tungsten_get_ports, which will display the running Tungsten processes and the associated TCP ports that those processes are listening on.

      Issues: CT-1970

    • The tungsten_get_ports command now supports the /opt/continuent/tungsten/tungsten-connector/conf/interfaces.json file, if found.

      Issues: CT-2003

    • The tungsten_get_ports command now use values based on values obtained from tpm query values when possible for the greatest output accuracy.

      Issues: CT-2008

Bug Fixes

  • Command-line Tools

    • Worked around a ruby issue where DNS name resolution was sometimes preferred to file-based resolution in TPM. The following logic now applies: first trying file-based host name resolution, upon failure try DNS, then through ping command

      Issues: CT-1653

  • Core Replicator

    • Fixed an issue in the applier to ensure it is not committing too often with a multi-service replicator (active/active for example, more specifically with AWS Aurora, or an unprivileged MySQL, backend).

      Issues: CT-2004

  • Filters

    • Fixed an issue where the dropsqlmodes filter would fail to remove invalid sql modes from a multi-statement event

      Issues: CT-1993

2.12. Tungsten Replicator 6.1.19 GA (17 Oct 2022)

Version End of Life. 15 Aug 2024

Release 6.1.19 contains a number of critical bug fixes and improvements, specifically for customers using Parallel Replication.

Behavior Changes

The following changes have been made to Tungsten Replicator and may affect existing scripts and integration tools. Any scripts or environment which make use of these tools should check and update for the new configuration:

  • Command-line Tools

    • The check_tungsten.sh was deprecated in release 6.1.18 and has now been removed from this release.

      Issues: CT-1939

  • Core Replicator

    • repl_svc_extractor_multi_frag_service_detection is now turned ON by default. Event shards are determined at extraction time. With fragmented events, the shard cannot be determined by only reading the first fragment, but needs to check the last fragment as well. With this setting turned OFF, there is no issue with pipelines that don't need it, i.e. no parallel apply downstream replicas. However, as this is done at extract time, THL contains this information, and adding or changing a replica using parallel apply could introduce issues.

      Note

      It can be disabled if you see a performance overhead but this should be done with caution. For Aurora<>Aurora Active/Active deployments it is essential that this property be left ON.

      Issues: CT-1959

Improvements, new features and functionality

  • Command-line Tools

    • The tpm command calls to glob have been improved to be more strict and compliant.

      Issues: CT-1940

    • The tpm ask stages and tpm ask allstages commands have been added to display the Replicator stages for the current node (stages) and the stages for each role (allstages).

      Issues: CT-1943

    • The tpm ask command has five new variables available: dsrole & dsstate for the current datasource, and trrole & trstate for the current replicator, and nodeinfo which displays all 4 of the new variables.

      Issues: CT-1944

Bug Fixes

  • Command-line Tools

    • Fixes an issue where the pause state would be badly displayed under no load, when the pause reaches the defined time. Note this is only a display issue : once an event is received, the pause state and displayed remaining time will be reseted correctly.

      Issues: CT-1942

    • The tpm mysql command will now gracefully handle being run on a non-database node.

      Issues: CT-1946

  • Core Replicator

    • Fixes an issue that would leave a transaction uncommitted longer than necessary.

      Issues: CT-1958

    • Fixed an issue where the shard_id stored in the trep_commit_seqno table would show an incorrect shard after processing a skipped event (an event from another channel)

      Issues: CT-1967

    • Fixed an issue where the replicator would hang after applying a DROP TABLE event, that originally failed on the primary, but got logged into the binlog.

      Issues: CT-1973

  • Filters

    • The BidiRemoteSlaveFilter could fail to correctly flag fragmented events in unprivileged environments (Aurora, for example) In such an environment (multi-active, unprivileged database access), a new setting was introduced to force extraction process to read ahead to the last fragment to detect the service name (false by default). Enabled with repl_svc_extractor_multi_frag_service_detection=true

      Issues: CT-1351

2.13. Tungsten Replicator 6.1.18 GA (7 Sept 2022)

Version End of Life. 15 Aug 2024

Release 6.1.18 contains a number of minor bug fixes and improvements.

Behavior Changes

The following changes have been made to Tungsten Replicator and may affect existing scripts and integration tools. Any scripts or environment which make use of these tools should check and update for the new configuration:

  • Command-line Tools

    • The check_tungsten.sh script has been deprecated and no longer maintained. It will be removed in the next patch release and users should use the newer tmonitor commands.

      Issues: CT-1038

Improvements, new features and functionality

  • Command-line Tools

    • The tungsten_merge_logs command now supports the --before TIMESTAMP and --after TIMESTAMP filters

      Issues: CT-1869

  • Core Replicator

    • The thl list command now displays an approximative field size in bytes for row-based replication.

      Issues: CT-1847

    • Added a new feature that enables pausing a replicator stage for some amount of time.

      This will pause the given stage for 100 seconds.:

      trepctl pause -stage thl-to-q -time 100

      This will pause the stage indefinitely (or until restart, etc) Add -y to avoid the prompt message whether you are sure.

      trepctl pause -stage thl-to-q

      For the previous 2 commands, running a pause command again will override the previous command.

      This will resume the suspended stage (Note that if the stage is not paused, this will have no effect):

      trepctl resume -stage thl-to-q

      Note

      Please note this pause does not survive a replicator restart or a service offline/online.

      Issues: CT-1912

  • Other Issues

    • General improvements in MySQL 8.0 support.

      Issues: CT-1346

Bug Fixes

  • Installation and Deployment

    • Fixes issues where fixed properties and filters passed to tpm in service stanzas were not being configured correctly

      Issues: CT-1463

  • Command-line Tools

    • replicator start offline would not properly pass the offline argument to systemd scripts when installed through deployall.

      Issues: CT-1836

    • undeployall script would not properly uninstall systemd services on some distributions including amazon linux.

      Issues: CT-1845

    • The tpm diag command now passes when the nodename defined in the tungsten.ini is the shortname, and DNS returns the FQDN.

      Issues: CT-1908

    • The tpm diag command now gathers the mysql.log file when SSL is enabled in the server

      Issues: CT-1933

    • The tpm diag command now behaves properly when the MySQL server is logging to STDERR.

      Issues: CT-1935

  • Backup and Restore

    • Fixed an issue where an xtrabackup generated by the replicator would fail to be restored using trepctl restore command.

      Issues: CT-1575

  • Core Replicator

    • Fixes an issue that would prevent a service from going offline at a specified time (trepctl online -until-time) when parallel apply is enabled. This is a rework of CT-1243.

      Issues: CT-1684

    • Fixed a possible issue when recovering an old primary as a replica after failover when parallel apply is enabled, that could lead the replica to be unable to come online and require a reprovisioning of this replica.

      Issues: CT-1890

    • Fixes an issue that prevented geometry datatypes with SRID from being replicated.

      Issues: CT-1904

    • Fixed an issue where filtered events would trigger a useless update to the service trep_commit_seqno table while it is overwritten anyway once the last statement of the applied event is done, just prior to committing the whole block.

      Issues: CT-1931

2.14. Tungsten Replicator 6.1.17 GA (16 May 2022)

Version End of Life. 15 Aug 2024

Release 6.1.17 contains a number of minor bug fixes and improvements.

Behavior Changes

The following changes have been made to Tungsten Replicator and may affect existing scripts and integration tools. Any scripts or environment which make use of these tools should check and update for the new configuration:

  • Command-line Tools

    • Changed output of thl purge command when no lower and upper bounds are given from 'Deleting events where' to 'Deleting all events'.

      Issues: CT-1738

    • The tpm diag command now gathers the output of the tpm ask summary command

      Issues: CT-1827

Improvements, new features and functionality

  • Command-line Tools

    • Added the ability to turn auto recovery on or off dynamically, removing the need to run tpm update.

      This is done by running the following command:

      shell> trepctl -service servicename setdynamic -property replicator.autoRecoveryMaxAttempts -value <number>

      Note

      The service must be offline before changing the property

      Issues: CT-1088

  • Core Replicator

    • Added a new property to allow configuration of character set used during batch loading into MySQL targets.

      Default is ut8mb4. Override using the following:

      property=replicator.applier.dbms.loadCharset=utf8

      Issues: CT-1647

  • Security

    • Replaced official log4j library with a secured version in which all vulnerable classes have been removed. This prevents exposing the software following a (user) misconfiguration of the log4j properties

      Issues: CT-1810

  • Other Issues

    • IPv6 host addresses are now fully supported.

      Can be enabled with the following configuration property:

      prefer-ip-stack="6"

      By default, IPv4 is enabled, which equates to the value of "4" in the above property.

      Issues: CT-1537

Bug Fixes

  • Installation and Deployment

    • When services are deployed with systemd and MySQL could not start due to an error, tpm would not be able to later start MySQL

      Issues: CT-1734

    • deployall script now properly displays the executable prefix for restarting services, typically mm_treplicator in Multi-Site/Active-Active replicator-only installs, as well as the full path to component binaries

      Issues: CT-1835

  • Command-line Tools

    • When MySql services where badly installed, some distribution could show a “not-found” status within systemctl, confusing tpm

      Issues: CT-1677

    • The tpm command now communicates properly when there is no INI configuration file or staging-method deploy.cfg configuration defined.

      Issues: CT-1712

    • tungsten_monitor.rb script no longer uses sudo to send emails if the configuration doesn't allow it.

      Issues: CT-1737

    • Improves the tpm diag command so that it waits 2 seconds for replicator thread dump to complete.

      Issues: CT-1792

    • tpm now properly reports errors upon timeout executing commands. Typically, when a host is down, when upgrading or installing, tpm will now properly report that pinging this host failed.

      Issues: CT-1819

  • Core Replicator

    • Due to a change in binlogging behavior from MySQL 5.7 onwards (Including Aurora) when using the standalone Tungsten Replicator to configure bi-directional replication with unprivileged access (ie unable to control binlogging behavior), certain DDL statements failed to be tagged correctly causing them to be re-applied to the originating source database, resulting in replicator errors.

      Issues: CT-1083

    • When connecting to a THL server, a client will now connect to the next available host in its THL uri, if the first does not have the sequence number that the client requires. The client will then fail only if none of the hosts from the uri can provide the needed sequence number.

      Issues: CT-1558

    • Fixed an issue when using Parallel apply that would show a NullPointerException in case an event could either not be found or be corrupted in THL. This will now display a correct message Missing or corrupted event from storage

      Issues: CT-1722

    • Fixed an issue with the batch loader where DATETIME and TIME datatypes with milliseconds would be replicated as .000 instead of the real millisecond value.

      Issues: CT-1739

    • Fixed an issue where trepctl was leaving JMX connections opened.

      Issues: CT-1752

    • Added more debug information for detecting possible hanging connections while a THL client connects to the THL server. Also, added socket timeout for the connection initialization

      Issues: CT-1760

    • Fixed the EnumToString and pkey filters to renew their database connections (every hour by default). This can be changed with the following properties:

      property=replicator.filter.enumtostring.reconnectTimeout=3600
      property=replicator.filter.pkey.reconnectTimeout=3600

      Issues: CT-1786

2.15. Tungsten Replicator 6.1.16 GA (20 Dec 2021)

Version End of Life. 15 Aug 2024

Release 6.1.16 contains a number of minor bug fixes and improvements.

Behavior Changes

The following changes have been made to Tungsten Replicator and may affect existing scripts and integration tools. Any scripts or environment which make use of these tools should check and update for the new configuration:

  • Command-line Tools

    • The Perl-based command-line tools now include the latest version of JSON::PP

      Issues: CT-1679

    • The tungsten_show_processlist command has been re-located to the cluster-home/bin from tools/.

      Issues: CT-1695

  • Documentation

    • Updated the open_source_licenses.txt file to include most up-to-date information for the included Perl modules.

      Issues: CT-1675

Improvements, new features and functionality

  • Command-line Tools

    • A new command, tungsten_merge_logs has been added which will gather Replicator logs into one file sorted by timestamp for easier troubleshooting.

      Issues: CT-1667

    • A new command tungsten_get_mysql_datadir has been added which will gather and display the actual running value for the mysql data directory (datadir) with any sym-links to the physical destination resolved.

      Issues: CT-1690

Bug Fixes

  • Command-line Tools

    • Now properly adds executable-prefix (when specified) to systemd boot scripts for the replicator.

      Issues: CT-1663

    • The tpm ask tool now works properly on all supported Linux distros.

      Issues: CT-1682

    • The tpm diag command now uses sudo to read the mysqld.log file.

      Issues: CT-1685

    • The tpm --help text now includes the --ini argument.

      Issues: CT-1693

  • Core Replicator

    • Fixes an issue where the replicator would loop forever if a wrong startup option was provided, e.g replicator start wrong

      Now, the replicator will stop with a fatal error message logged Unrecognized option: wrong - Shutting down

      Issues: CT-1672

    • Fixes a replication issue with some JSON fields that would contain a single quote within the JSON text

      Issues: CT-1691

  • Documentation

    • The documentation for the tpm command now describes the use of the --ini argument fully.

      Issues: CT-1692

2.16. Tungsten Replicator 6.1.15 GA (19 Oct 2021)

Version End of Life. 15 Aug 2024

Release 6.1.15 contains a number of minor bug fixes and improvements.

Improvements, new features and functionality

  • Core Replicator

    • Added ability to the Batch Applier, to apply to a remote MySQL instance using LOAD DATA LOCAL INFILE.

      This is disabled by default, and can be enabled by the following property:

      property=replicator.applier.dbms.useLoadDataLocalInfile=true

      Issues: CT-1622

Bug Fixes

  • Command-line Tools

    • The tpm update command now handles the --replace-release argument properly when --ini is also specified.

      Issues: CT-1623

    • Improved Perl-based command-line tools to use routines with no external dependencies in place of Data::Dumper

      Issues: CT-1630

  • Core Replicator

    • Improved error reporting when the replicator is not able to open a file. This was causing a NullPointerException to be raised.

      Issues: CT-1618

  • Filters

    • Fixed an issue with the ColumnNameFilter that was not resetting its internal connection to the database.

      This would then fail if the database connection would time out (e.g. MySQL wait_timeout reached). The connection is now renewed every 1 hour by default.

      This can be reconfigured to a different value by adding the following tpm property:

      property=replicator.filter.colnames.reconnectTimeout=<time in seconds>

      Issues: CT-1646

2.17. Tungsten Replicator 6.1.14 GA (17 Aug 2021)

Version End of Life. 15 Aug 2024

Release 6.1.14 contains a number of bugs fixes and improvements to the Batch loader for MySQL. In addition, support for XA transactions is now available.

Improvements, new features and functionality

  • Command-line Tools

    • Added tpm support for tuning maxDelayInterval, used to tune parallel apply. Usage example:

      property=replicator.store.parallel-queue.maxDelayInterval=120

      This sets the max delay interval to 2 minutes

      Issues: CT-1541

    • A new tpm flag has been added:

      • svc-systemd-config-replicator

      This flag can be used to provide a custom systemd configuration that will be used in place of the default, generated one.

      This is typically useful when ExecStartPre or ExecStartPost commands are needed

      Issues: CT-1555

    • Improved the tpm command's internal logic and introduced a new sub-command, tpm ask.

      Issues: CT-1573

    • Improved the check_tungsten_online command:

      • determine the cluster name by default if none is provided via -s

      • always check ls resources even if a dataservice name is provided

      • add an option to be node-specific, ignoring any status from other nodes

      Issues: CT-1578

    • New ddlscan template (ddl-mysql-staging.vm) available to generate ddl required for staging tables when deploying the batch applier on a MySQL target.

      Issues: CT-1587

  • Backup and Restore

    • A new tpm option has been added --backup-options

      This property can be used to pass options directly to the backup binary (Xtrabackup, mariabackup etc). This can be especially useful when the xtrabackup and MySQL versions differ. Generally, MySQL is always ahead of xtrabackup and in some situations xtrabackup will fail if the underlying MySQL patch release does not match the xtrabackup patch release. In this scenario, you could utilise the new option to force xtrabackup to not check the versions, for example:

      backup-options=--no-server-version-check

      Multiple options can be passed as a space separated list, for example.

      backup-options=--option1 --option2 --option3

      Note

      Options passed are specific to the underlying binary and are not validated by Tungsten, therefore you must ensure you only use options that are valid, and check syntax accurately.

      Issues: CT-1595

  • Core Replicator

    • Support for XA transactions has been added.

      Issues: CT-925

    • When using the MySQL Batch Applier, you can now choose to drop all deletes by specifying the following tpm property:

      property=replicator.applier.dbms.skipDeletes=true

      This is disabled (false) by default.

      Issues: CT-1588

    • When using the MySQL Batch Applier, you can now choose to log rows into an error table that would otherwise have either generated a duplicate key error.

      This is disabled by default, but can be enabled and configured by using the following two new properties:

      property=replicator.applier.dbms.errorTablePrefix=error_xxx_ 
      property=replicator.applier.dbms.batchLogDuplicateRows=true

      In addition, if your tables contain Foreign Keys, you can choose to disable them during the data loading process using the following property:

      property=replicator.applier.dbms.disableForeignKeys=true

      Issues: CT-1589

Bug Fixes

  • Command-line Tools

    • tpm was failing to check application password when single quoted.

      Issues: CT-1257

    • The tpm update command now properly parses the --ini argument.

      Issues: CT-1551

2.18. Tungsten Replicator 6.1.13 GA (27 May 2021)

Version End of Life. 15 Aug 2024

Release 6.1.13 contains a number of bug fixes, and contains updated drivers for replicating to newer releases of PostgreSQL.

Behavior Changes

The following changes have been made to Tungsten Replicator and may affect existing scripts and integration tools. Any scripts or environment which make use of these tools should check and update for the new configuration:

  • Command-line Tools

    • Ensure that all tools are available in the appropriate software packages.

      Issues: CT-1518

Improvements, new features and functionality

  • Command-line Tools

    • A new tpm option delete-service is now available to simplify the removal of replicator services.

      Issues: CT-327

    • A new utility, tungsten_skip_seqno has been added which allows events to be skipped based on filters, allowing the Tungsten Replicator to come back online with less manual intervention.

      Issues: CT-42, CT-1087, CT-1453, CT-1511

  • Heterogeneous Replication

    • Upgraded PostgreSQL JDBC driver to the latest release to allow compatibility with PostgreSQL 12.

      Issues: CT-1521

Bug Fixes

  • Command-line Tools

    • This release includes a wide range of improvements and bug fixes for tpm and other command-line tools.

      Issues: CT-1509

    • Fixes an issue where the undeployall command would create a root owned wrapper.log in the ./tools directory.

      Issues: CT-1516

    • Fixed a tpm call to systemd services listing which was making tpm wrongly fall back to init.d. This issue was found when testing MariaDB 10.0.

      Issues: CT-1524

    • tpm now supports new MariaDB my_print_defaults flag --defaults-file

      Issues: CT-1528

  • Core Replicator

    • Fixes an issue where null values in JSON structures were incorrectly quoted once replicated.

      For example :

      [{"example":null}]

      would be replicated as

      [{"example":"null"}]

      Issues: CT-1523

2.19. Tungsten Replicator 6.1.12 GA (8 Mar 2021)

Version End of Life. 15 Aug 2024

Release 6.1.12 is a minor bug fix release.

Improvements, new features and functionality

  • Monitoring

    • Support has now been added to allow monitoring via the New Relic Insights API, using a new script tungsten_newrelic_event

      Issues: CT-892

Bug Fixes

  • Command-line Tools

    • tpm install would fail on FIPS enabled enviroments.

      Issues: CT-1451

  • Core Replicator

    • A new mapcharset filter is included, which fixes collation incompatibilities when replicating from MySQL 8 to an older release (5.7 or 5.6)

      This new filter should be enabled within any MySQL applier where the source is generated from a MySQL 8 database and the target is MySQL 5.7 or lower.

      Issues: CT-1404

    • If SSL is enabled between replicators for THL Transfer, and a source node fails and becomes unresponsive, or is slow/overloaded, the replicator on the replica can fail to go offline due to a timeout, remaining in a GOING-ONLINE:SYNCHRONIZING state.

      Issues: CT-1459

2.20. Tungsten Replicator 6.1.11 GA (21 Jan 2021)

Version End of Life. 15 Aug 2024

Release 6.1.11 is a major bug fix release specifically for clustering deployments.

This Tungsten Replicator release contains no changes, but was released as part of Continuents Standard release process to maintain consistency with version numbers.

2.21. Tungsten Replicator 6.1.10 GA (15 Dec 2020)

Version End of Life. 15 Aug 2024

Release 6.1.10 is a minor bug fix release.

Improvements, new features and functionality

  • Command-line Tools

    • tpm diag now collects routing table information via route -n , and has two new command-line arguments: --include and --groups.

      --include specifies a comma-separated list of subroutines to include. Any gather subroutine not listed will be skipped.

      --groups specifies a comma-separated list of subroutine groups to include. Any group not listed will be skipped.

      Issues: CT-1399

2.22. Tungsten Replicator 6.1.9 GA (23 Nov 2020)

Version End of Life. 15 Aug 2024

Release 6.1.9 is a critical bug fix release containing a fix for a critial bug within the Replicator related to the handling of Timezone.

Bug Fixes

  • Command-line Tools

    • tpm update no longer fails when using the staging method to upgrade to a new version.

      Issues: CT-1381

  • Core Replicator

    • In some edge case scenarios, the replicator was not setting the session time_zone correctly when proceeding sessions had a different time_zone applied, this could lead to situations where TIMESTAMP values could be applied into replica nodes with an incorrect time_zone offset applied.

      Issues: CT-1390

2.23. Tungsten Replicator 6.1.8 GA (2 Nov 2020)

Version End of Life. 15 Aug 2024

Release 6.1.8 is a minor bug fix release.

Bug Fixes

  • Core Replicator

    • Fixes an issue that would prevent a service from going offline at a specified time (trepctl online -until-time) or at a specific seqno (trepctl online -until-seqno) when parallel apply is enabled.

      Issues: CT-1243

    • In MySQL releases using old row events format (MySQL 5.6 or earlier), Delete_rows_v1 were badly handled, leading to an extraction error when handling such an event type.

      Issues: CT-1358

2.24. Tungsten Replicator 6.1.7 GA (5 Oct 2020)

Version End of Life. 15 Aug 2024

Release 6.1.7 was a minor bug fix release containing a fix for SSL Communications specific to clustering deployments.

This Tungsten Replicator release contains no new changes, but was released as part of Continuents Standard release process to maintain consistency with version numbers.

2.25. Tungsten Replicator 6.1.6 GA (20 Aug 2020)

Version End of Life. 15 Aug 2024

Release 6.1.6 is a minor bug fix containing a fix for bi-directional standlone Replicator deployments.

Bug Fixes

  • Core Replicator

    • Allows multiple service names to be supplied to property=local.service.name when configuring bi-directional replication between a Composite Active/Active cluster topology and a MySQL target to prevent loopback of transactions.

      Issues: CT-1308

2.26. Tungsten Replicator 6.1.5 GA (5 Aug 2020)

Version End of Life. 15 Aug 2024

Release 6.1.5 is a small interim bug fix with a number of issues resolved within the Core Replicator, sepcifically for heterogeneous environments.

Behavior Changes

The following changes have been made to Tungsten Replicator and may affect existing scripts and integration tools. Any scripts or environment which make use of these tools should check and update for the new configuration:

  • Heterogeneous Replication

    • When using the batch applier in Heterogeneous pipelines (Redshift, Vertica, Hadoop) the batch applier removes DDL statements.

      There may be occasions when you intentionally want DDL to pass through, such as if you have a custom filter that injects custom DDL statements into the pipeline, however the batch applier would always remove them.

      A new property is now available to control this behaviour. Set property=replicator.applier.dbms.applyStatements=true to allow the batch applier to retain DDL statements. The default value of false retains the original behaviour of removing DDL.

      Issues: CT-1270

Known Issues

The following issues are known within this release but not considered critical, nor impact the operation of Tungsten Replicator. They will be addressed in a subsequent patch release.

  • Core Replicator

    • In MySQL release 8.0.21 the behavior of CREATE TABLE ... AS SELECT ... has changed, resulting in the transactions being logged differenly in the binary log. This change in behavior will cause the replicators to fail.

      Until a fix is implemented within the replicator, the workaround for this will be to split the action into separate CREATE TABLE ... followed by INSERT INTO ... SELECT FROM... statements.

      If this is not possible, then you will need to manually create the table on all nodes, and then skip the resulting error in the replicator, allowing the subsequent loading of the data to continue.

      Issues: CT-1301

Bug Fixes

  • Core Replicator

    • When replicating data that included timestamps, the replicator would update the timestamp value to the time within the commit from the incoming THL. When using statement based replication times would be correctly replicated, but if using a mixture of statement and row based replication, the timestamp value would not be set back to the default time when switching between statement and row based events. This would not cause problems in the applied host, except when log_slave_updates was enabled. In this case, all row-based events after a statement based event would have the same timestamp value applied.

      This was most commonly seen when using the standalone replicator to replicate into a Cluster, either from a standlone source, or a cluster source.

      Issues: CT-1255

    • For offboard extraction, the replicator would appear to be ONLINE but not actually processing new events.

      This is due to the relay client getting an incomplete packet from the remote database and going into a WAITING state.

      To handle these situations, a new property has been included that will set a timeout and if the replicator does not process an event in the given timeout period, we assume we have lost the link to the remote database and will place the replicator into an OFFLINE:ERROR state.

      Providing auto-recovery has been enabled using the auto-recovery-max-attempts parameter, the replicator will then restart and proceed successfully.

      The new property to include is property=replicator.extractor.dbms.relayLogIdleTimeout

      The default value (0) will disable the timeout. Values provided are in seconds, so 300 would be 5 minutes.

      Setting the timeout too low in quieter systems may result in unnecessary replicator restarts. The value should be set according to the activity levels of your database. If the source is very active with constant updates, then a low value would be appropriate. On quieter systems that may have long periods of inactivty, should have a timeout value set no less than the longest, normal, period of inactivity within your system.

      Issues: CT-1262

    • If filtering is in use, and a space appeared either side of the delimiter in a "schema.table" definition in your SQL, the replicator would fail to parse the statement correctly.

      For example, all of the examples below are valid SQL but would cause a failure in the replicator:

      sql> CREATE TABLE myschema. mytable (....
      
      sql> CREATE TABLE myschema .mytable (....
      
      sql> CREATE TABLE myschema . mytable (....

      Issues: CT-1278

    • Fixes a bug in the Drizzle Driver whereby a failing prepared statement that exceeds 1000 characters would report a String index out of range: 999 error rather than the actual error.

      Issues: CT-1303

2.27. Tungsten Replicator 6.1.4 GA (4 June 2020)

Version End of Life. 15 Aug 2024

Release 6.1.4 contains a number of improvements and bug fixes, specifically for the tpm command line tool and improvements to the Redshift Applier. In addition this release now fully supports the latest binlog compression feature of MySQL 8.0.20.

Improvements, new features and functionality

  • Command-line Tools

    • Improves tpm performance by using more efficient routines to calculate paths.

      Issues: CT-1168

    • Added the ability for tpm diag to skip both individual gather subroutines along with entire groups of gather subroutines.

      Also added ability to list all gather groups and subroutines using --list for use with the --skip and --skipgroups cli arguments.

      Issues: CT-1176

    • tungsten_provision_slave has been rewritten fixing a number of issues in the previous release. This version was previously released as the Beta script prov-sl.sh.

      Issues: CT-1208

    • A number of performance improvements and fixes have been incorporated into the Drizzle Driver used for communication between components and MySQL, these include:

      • Better handling of large queries close to max network packet size.

      • Batch Support. Instead of sending statements one by one, the driver will be able to send multiple statements at once, avoiding round trips between the driver and MySQL server.

      • Fixes issues with interpreting useSSL on connect string URLs.

      Issues: CT-1215, CT-1216, CT-1217, CT-1228

    • The tungsten_send_diag command now accepts arguments for the tpm diag command using --args or -a for short.

      You must enclose the arguments in quotes, for example:

      shell> tungsten_send_diag -c 9999 -d --args ‘--all -v’

      Issues: CT-1220

  • Core Replicator

    • debug has been disabled by default in the schemachange filter. Resulting in reduced noise in the replicator log file.

      Issues: CT-1013

    • A new replicator role thl-client has been added. This new role allows the replicator to download THL from a Primary, but not apply to the target database.

      Issues: CT-1123

    • A new delayInMs filter has been added which allows the applying of THL to a Replica to be delayed. The filter allows millisecond precision. This filter works in the same way as the TimeDelayFilter, however that filter only allow second precision.

      Issues: CT-1191

    • A new droprow JavaScript filter has been added.

      The filter works at ROW level and allows the filtering out of rows based on one or more column/value matches

      Issues: CT-1213

    • When configuring the Redshift applier, you can now configure which tool the applier will use for posting CSV files to S3. Options are s3cmd (default), s4cmd or aws.

      Issues: CT-1218

    • A number of improvements have been made to the Redshift Applier to allow optional levels of table locking.

      This is particularly useful when you have multiple Redshift Appliers in a Fan-In topology, and/or very high volumes of data to process.

      The additional locking options reduce the risk of Redshift Serializable Isolation Violation errors occuring.

      Full details of how to utilise the new options can be read at Handling Concurrent Writes from Multiple Appliers

      Issues: CT-1221

    • Tungsten Replicator can now correctly extract and parse Binary Log entries when the MySQL option binlog-transaction-compression has been enabled.

      binlog-transaction-compression is a new parameter introduced from MySQL 8.0.20.

      Issues: CT-1223

Bug Fixes

  • Command-line Tools

    • In certain edge cases, tungsten_provision_slave would fail to detect if mysql was shutdown.

      Issues: CT-1096

    • tpm diag now collects directories specified with !includedir in the /etc/my.cnf file.

      Issues: CT-1153

    • Fixes the tpm update command, which would exit with the error:

      Argument " (error code 1)" isn't numeric

      Issues: CT-1165

    • tpm diag now collects any files specified by !include directives in the /etc/my.cnf and /etc/mysql/my.cnf files.

      tpm diag also looks in /etc/mysql/my.cnf for !includedir directives

      Issues: CT-1169

    • Fixes a bug which prevented tungsten_send_diag from uploading a self-generated diagnostic zip file.

      Issues: CT-1170

    • tpm diag now properly derives the correct target path to the releases directory if the home directory in the configuration points to a sym-link.

      Issues: CT-1172

    • Removed tpm diag call to sudo for gathering ifconfig and lsb_release commands.

      Issues: CT-1175

    • tpm update would fail and report errors if deployall had been executed.

      Issues: CT-1179

    • tpm diag no longer requires the mysql command-line client when running on non-MySQL Applier nodes, and no longer attempts to gather any MySQL diagnostic information.

      Issues: CT-1188

    • Removes the requirement to execute component start/stop commands with sudo when called through systemd.

      This is specific to start/stop actions following the use of the deployall scripts.

      Issues: CT-1193

    • Fixes cases where tpm fails when the OS hostname command returns a different string than what is used in the configuration (i.e. hostname returns a FQDN, yet the configuration contains shortnames like db1, db2, etc.).

      Issues: CT-1206

    • In certain cases, after a reprovision, tungsten_provision_slave didn’t always run the steps to reset the local replicator service. This made the replicator go into an error state after provision had completed.

      Issues: CT-1210

    • The tpm diag command now handles the cluster-slave topology more gracefully, and properly handles cluster nodes without the Connector installed.

      Improved output text clarity by converting multiple verbose-level outputs to debug, and warnings to notice-level.

      Issues: CT-1222

    • tpm diag now gathers sym-linked files correctly.

      Issues: CT-1232

    • ddlscan now sets the datatype for sequence number columns to BIGINT when generating staging table DDL for Redshift deployments.

      Issues: CT-1235

    • Fixes a situation where tpm update exits with a Data::Dumper error.

      Issues: CT-1249

  • Core Replicator

    • In heterogeneous replicator deployments, the convertstringfrommysql filter would fail to convert strings for alphanumeric key values.

      Issues: CT-1128

    • Tungsten Replicator could wrongly think it is already in DEGRADED mode when trying to put it into DEGRADED state.

      Issues: CT-1131

    • Tungsten Replicator now recognises Amazon AWS SSL Certificates to enable SSL communication with AWS Aurora.

      Issues: CT-1173

    • When host server time (and thus MySQL time) is not configured as UTC, issuing cluster heartbeat or trepctl heartbeat in the first hours around daylight savings time would create an invalid time in MySQL.

      For more information on timezones when issuing heartbeats, see trepctl heartbeat Time Zone Handling

      Issues: CT-1174

    • The replicator would fail to apply into Vertica when configured as an offboard installation due to the applier incorrectly expecting the csv files to exist locally on the remote Vertica host.

      Issues: CT-1194

    • Due to a change in the Binary log structure introduced in MySQL 8.0.16, the replicator would fail to extract transactions for Partitioned tables.

      Issues: CT-1201

    • The replicator would fail to correctly parse statements with leading comment blocks in excess of 200 characters.

      Issues: CT-1236

2.28. Tungsten Replicator 6.1.3 GA (17 February 2020)

Version End of Life. 15 Aug 2024

Release 6.1.3 contains a small number of improvements and fixes to common command line tools, and introduces compatibility with MongoDB Atlas.

Behavior Changes

The following changes have been made to Tungsten Replicator and may affect existing scripts and integration tools. Any scripts or environment which make use of these tools should check and update for the new configuration:

  • Command-line Tools

    • tpm diag has been updated to provide additional feedback detailing the hosts that were gathered during the execution, and also provides examples of how to handle failures

      When running on a single host configured via the ini method:

      shell> tpm diag
      Collecting localhost diagnostics.
      Note: to gather for all hosts please use the "-a" switch and ensure that you have paswordless »
      ssh access set between the hosts.
      Collecting diag information on db1.....
      Diagnostic information written to /home/tungsten/tungsten-diag-2020-02-06-19-34-25.zip

      When running on a staging host, or with the -a flag:

      shell> tpm diag [-a|--allhosts]
      Collecting full cluster diagnostics
      Note: if ssh access to any of the cluster hosts is denied, use "--local" or "--hosts=<host1,host2,...>"
      Collecting diag information on db1.....
      Collecting diag information on db2.....
      Collecting diag information on db3.....
      Diagnostic information written to /home/tungsten/tungsten-diag-2020-02-06-19-34-25.zip

      Issues: CT-1137

Bug Fixes

  • Command-line Tools

    • tpm would fail to run on some Operating Systems due to missing realpath

      tpm has been changed to use readlink which is commonly installed by default on most operating systems, however if it is not available, you may be required to install GNU coreutils to satisfy this dependency

      Issues: CT-1124

    • Removed dependency on perl module Time::HiRes from tpm

      Issues: CT-1126

    • Added support for handling missing dependency (Data::Dumper) within various tpm subcommands

      Issues: CT-1130

    • tpm will now work on MacOS/X systems, provided greadlink is installed.

      Issues: CT-1147

    • tpm install will no longer report that the linux distribution cannot be determined on SUSE platforms.

      Issues: CT-1148

    • Fixes a condition where tpm diag would fail to set the working path correctly, especially on Debian 8 hosts.

      Issues: CT-1150

    • tpm diag now checks for OS commands in additional paths (/bin, /sbin, /usr/bin and /usr/sbin)

      Issues: CT-1160

    • Fixes an issue introduced in v6.1.2 where the use of the undeployall script would stop services as it removed them from systemctl control

      Issues: CT-1166

  • Core Replicator

    • The MongoDB Applier has been updated to use the latest MongoDB JDBC Driver

      Issues: CT-1134

    • The MongoDB Applier now supports MongoDB Atlas as a target

      Issues: CT-1142

    • The replicator would fail with Unknown column '' in 'where clause when replicating between MySQL 8 hosts where the client application would write data into the source database host using a different collation to that of the default on the target database.

      The replicator would fail due to a mismatch in these collations when querying the information_schema.columns view to gather metadata ahead of applying to the target

      Issues: CT-1145

2.29. Tungsten Replicator 6.1.2 GA (20 January 2020)

Version End of Life. 15 Aug 2024

Release 6.1.2 contains both significant improvements as well as some needed bugfixes.

Behavior Changes

The following changes have been made to Tungsten Replicator and may affect existing scripts and integration tools. Any scripts or environment which make use of these tools should check and update for the new configuration:

  • Behavior Changes

    • Certified the Tungsten product suite with Java 11.

      A small set of minor issues have been found and fixed (CT-1091, CT-1076) along with this certification.

      The code is now compiled with Java compiler v11 while keeping Java 8 compatibility.

      Java 9 and 10 have been tested and validated but certification and support will only cover Long Term releases.

      Note

      Known Issue

      With Java 11, command line tools are slower. There is no impact on the overall clustering or replication performance but this can affect manual operations using CLI tools such as cctrl and trepctl

      Issues: CT-1052

Improvements, new features and functionality

  • Command-line Tools

    • The tpm command was originally written in Ruby. This improvement converts tpm to Perl over time, starting with the tpm shell wrapper and refactoring each sub-command one-by-one.

      For this release, we have redone the diag and mysql sub-commands.

      Issues: CT-1048

  • Core Replicator

    • A new Replicator role, thl-server, has been added.

      This new feature allows your Replica replicators to still pull generated THL from a Primary even when the Primary replicator has stopped extracting from the binlogs.

      If used in Tungsten Clustering, this feature must only be enabled when the cluster is in MAINTENANCE mode.

      Issues: CT-58

      For more information, see Understanding Replicator Roles.

    • A new JavaScript filter dropddl.js has been added to allow selective removal of specific object DDL from THL.

      Issues: CT-1092

Bug Fixes

  • Behavior Changes

    • If you need to reposition the extractor, there are a number of ways to do this, including the use of the options -from-event or -base-seqno

      Both of these options are mutually exclusive, however in some situations, such as when positioning against an Aurora source, you may need to issue both of these options together. Previously this was not possible. In this release both options can now be supplied providing that you include the additional -force option, for example

      shell> trepctl -service serviceName online -base-seqno 53 -from-event 000412:762897 -force

      Issues: CT-1065

    • When the Replicator inserts a heartbeart there is an associated timezone. Previously, the heartbeat would be inserted using the GMT timezone, which fails during the DST switch window. The new default uses the Replicator host's timezone instead.

      This defaults change corrects an edge case where inserting a heartbeat will fail during the DST switch window when the MYSQL server is running in a different timezone than the Replicator (which runs in GMT).

      For example, on 31th March 2019, the time switch occurred @ 2AM in the Europe/Paris timezone. When inserting a heartbeat in the window between 4 and 5 AM (say at 4:15am), the corresponding GMT time would be 2:15am, which is invalid in the Europe/Paris timezone. Replicator would then fail if the MySQL timezone was set to Europe/Paris, as it would try to insert an invalid timestamp.

      A new option, -tz has been added into the trepctl heartbeat command to force the use of a specific timezone.

      For example, use GMT as the timezone when inserting a heartbeat:

      shell> trepctl heartbeat -tz NONE

      Use the Replicator host's timezone to insert the heartbeat:

      shell> trepctl heartbeat -tz HOST

      Use the given timezone to insert the heartbeat:

      shell> trepctl heartbeat -tz {valid timezone id}

      If the MySQL server timezone is different from the host timezone (which is strongly not recommended), then -tz {valid timezone id} should be used instead where {valid timezone id} should be the same as the MySQL server timezone.

      Issues: CT-1066

    • Corrected resource leak when loading Java keystores

      Issues: CT-1091

  • Command-line Tools

    • Fixed error message to indicate the need to specify a service on Composite Active/Active clusters for the tungsten_find_position and tungsten_find_seqno commands.

      Issues: CT-1098

    • The tpm command no longer reports warnings about existing system triggers with MySQL 8+

      Issues: CT-1099

  • Core Replicator

    • When configuring a Kafka Applier, the Kafka Port was set incorrectly

      Issues: CT-693

    • If a JSON field contained a single quote, the replicator would break during the apply stage whilst running the generated SQL into MySQL.

      Single quotes will now be properly escaped to solve this issue

      Issues: CT-983

    • Under rare circumstances (network packet loss or MySQL Server hang), the replicator would also hang until restarted.

      This issue has been fixed by using specific network timeouts in both the replicator and in the Drizzle jdbc driver connection logic

      Issues: CT-1034

    • When configuring Active/Active, standalone replicators, with the BidiSlave filter enabled, the replicator was incorrectly parsing certain DDL Statements and marking them as unsafe, as a result they were being dropped by the applier and ignored

      The full list of DDL commands fixed in this release are as follows:

      • CREATE|DROP TRIGGER

      • CREATE|DROP FUNCTION

      • CREATE|DROP|ALTER|RENAME USER

      • GRANT|REVOKE

      Issues: CT-1084, CT-1117

    • The following warnings would appear in the replicator log due to GTID events not being handled.

      WARN extractor.mysql.LogEvent Skipping unrecognized binlog event type 33, 34 or 35)

      The WARN message will no longer appear, however GTID Events are still not handled in this release, but will be fully extracted in a future release.

      Issues: CT-1114

2.30. Tungsten Replicator 6.1.1 GA (28 October 2019)

Version End of Life. 15 Aug 2024

Release 6.1.1 contains both significant improvements as well as some needed bugfixes.

Improvements, new features and functionality

  • Core Replicator

    • Added Clickhouse applier support.

      Issues: CT-383

    • If using the dropcolumn filter during extraction, in conjunction with the Batch Applier (eg Replicating to Redshift, Hadoop, Vertica) writes would fail with a CSV mismatch error due to gaps in the THL Index.

      However, for JDBC appliers, the gaps are required to ensure the correct column mapping

      To handle the two different requirements, a new property has been added to the filter to control whether or not to leave the THL index untouched (the default) or to re-order the Index ID's

      If applying to Batch targets, then the following property should be added to your configuration. The property is not required for JDBC targets.

      --property=replicator.filter.dropcolumn.fillGaps=true

      Issues: CT-1025

Bug Fixes

  • Command-line Tools

    • Fixed an issue that would prevent reading remote binary logs when using SSL.

      Issues: CT-958

    • Fixed an issue where the command trepctl -all-services status -name watches fails.

      Issues: CT-977

    • Restored previously-removed log file symbolic links under $CONTINUENT_ROOT/service_logs/

      Issues: CT-1026

    • Fixed a bug where tpm diag would generate an empty zip file if the hostnames contain hyphens (-) or periods (.)

      Issues: CT-1032

    • Improve ability to find needed binaries for commands: tungsten_find_position, tungsten_find_seqno and tungsten_get_rtt

      Issues: CT-1054

2.31. Tungsten Replicator 6.1.0 GA (31 July 2019)

Version End of Life. 15 Aug 2024

Release 6.1.0 contains both significant improvements as well as some needed bugfixes. One of the main features of this release is MySQL 8 support.

Improvements, new features and functionality

  • Command-line Tools

    • Two new utility scripts have been added to the release to help with setting the Replicator position:

      - tungsten_find_position, which assists with locating information in the THL based on the provided MySQL binary log event position and outputs a dsctl set command as output.

      - tungsten_find_seqno, which assists with locating information in the THL based on the provided sequence number and outputs a dsctl set command as output.

      Issues: CT-934

  • Core Replicator

    • A new, beta-quality command has been included called prov-sl.sh which is intended to eventually replace the current tungsten_provision_slave script.

      Currently, prov-sl.sh supports provisioning Replicas using mysqldump and xtrabackup tools, and is MySQL 8-compatible. 

      The prov-sl.sh command is written in Bash, has less dependencies compared to the current version and is meant to fix a number of issues with the current version.

      Backups are streamed from source to target so that an intermediate write to disk is not performed, resulting in faster provisioning times.

      Logs are written to $CONTINUENT_ROOT/service_logs/prov-sl.log (i.e. /opt/continuent/service_logs/prov-sl.log).

      For example, provision a Replica from [source db] using mysqldump (default):

      shell> prov-sl.sh -s {source db}

      As another example, use xtrabackup for the backup method, with 10 parallel threads (default is 4), and ssh is listening on port 2222:

      shell> prov-sl.sh -s {source db} -n xtrabackup -t 10 -p 2222

      Warning

      At the moment, prov-sl.sh does not support Composite Active/Active topologies when used with Tungsten Clustering, however it will be included in a future release.

      Issues: CT-614, CT-723, CT-809, CT-855, CT-963

    • Upgraded the Drizzle driver to support MySQL 8 authentication protocols (SHA256, caching_sha2).

      Issues: CT-914, CT-931, CT-966

    • The Redshift Applier now allows AWS authentication using IAM Roles. Previously authentication was possible via Access and Secret Key pairs only.

      Issues: CT-980

      For more information, see Redshift Preparation for Amazon Redshift Deployments.

Bug Fixes

  • Command-line Tools

    • When executing mysqldump, all Tungsten tools no longer use the --add-drop-database flag as it will prevent MySQL 8+ from restoring the dump.

      Issues: CT-935

    • Fixed a bug where tpm diag would generate an empty zip file if the hostnames contain hyphens (-) or periods (.)

      Issues: CT-1032

  • Core Replicator

    • Added support for missing charset GB18030 to correct WARN extractor.mysql.MysqlBinlog Unknown charset errors.

      Issues: CT-915, CT-932

    • Loading data into Redshift would fail with the following error if a row of data contained a specific control character (0x00 (NULL))

      Missing newline: Unexpected character 0x30 found at location nnn

      Issues: CT-984

    • Now properly extracting the Geometry datatype.

      Issues: CT-997

    • The ddl_map.json file used by the apply_schema_changes filter was missing a rule to handle ALTER TABLE statements when replicating between MySQL and Redshift

      Issues: CT-1002

    • The extract_schema_change filter wasn't escaping " (double-quotes) and the generated JSON would then cause the applier to error with

      pendingExceptionMessage: SyntaxError: missing } after property list »
      (../../tungsten-replicator//support/filters-javascript/apply_schema_changes.js#236(eval)#1)

      Issues: CT-1011

3. Tungsten Dashboard Release Notes

3.1. Tungsten Dashboard 1.0.15 GA (14 February 2024)

Version End of Life. 14 February 2025

Tungsten Dashboard provides a web-based UI for monitoring and managing Tungsten Clustering deployments.

Tungsten Dashboard v1.0.15 is a feature and bugfix release.

  • New Feature - Dashboard now checks to see if the pipelinesource on a replica is not pointing to the current primary and highlights the pipelineSource with bold red if mismatched.

  • New Feature - now able to change the Auto-Configure host and port via Settings

  • Improvement - Dashboard now has Connector polling disabled by default to reduce manager load. Enable via the Settings panel.

    Technical Details - automatically adds the ?includeRouters=false flag to the end of calls to /api/v2/manager/cluster/status and /api/v2/manager/status/service/{$service}

  • Improved Docker support.

  • Improved support for Distributed Data Groups (DDG), including an added setting for enableDDGNodeColors.

  • Improvement - updated various wording to differentiate between the Auto-Define and Auto-Confgigure features. updates all copyrights to 2024

  • Improvement - Updates all copyrights to 2024.

  • BugFix - Primary definition corrected to exclude shunned or failed masters.

  • BugFix - Settings panel help broken when fancy tooltips were not enabled.

  • BugFix - Auto-Define now gets the proper security defaults when none are specified in the form.

3.2. Tungsten Dashboard 1.0.14 GA (11 April 2023)

Version End of Life. 11 April 2024

Tungsten Dashboard provides a web-based UI for monitoring and managing Tungsten Clustering deployments.

Tungsten Dashboard v1.0.14 mainly provides support for jQuery 3.6.4 along with a new "Client Request Tracking" feature.

  • Upgraded to jQuery 3.6.4

  • Added new feature "Client Request Tracking" to log one line per client request call.

  • Improved formatting for the diagnostic upload result message.

3.3. Tungsten Dashboard 1.0.13 GA (31 January 2023)

Version End of Life. 31 January 2024

Tungsten Dashboard provides a web-based UI for monitoring and managing Tungsten Clustering deployments.

Tungsten Dashboard v1.0.13 is a release for a single bugfix.

  • Fixed an auto-configure regression for composite clusters.

3.4. Tungsten Dashboard 1.0.12 GA (14 December 2022)

Version End of Life. 14 December 2023

Tungsten Dashboard provides a web-based UI for monitoring and managing Tungsten Clustering deployments.

Tungsten Dashboard v1.0.12 mainly provides support for the new datasource drain feature in Tungsten Clustering v7.0.2, along with a few improvements and bugfixes.

  • The new datasource drain feature in Tungsten Clustering v7.0.2 is now supported in the Tungsten Dashboard node menus

    There is a new,related setting called drainTimeout which controls the length of time to wait before closing the connection to a database node from a Connector

  • Changed managerPort setting default value to 8201 from 8091

  • Updated the auto-refresh browser-specific timestamp display to fit better on the navbar

  • Corrected a page reload issue

3.5. Tungsten Dashboard 1.0.11 GA (8 November 2022)

Version End of Life. 8 November 2023

Tungsten Dashboard provides a web-based UI for monitoring and managing Tungsten Clustering deployments.

Tungsten Dashboard v1.0.11 provides a number of new features, improvements and bugfixes.

Cluster Configuration File Changes

  • The config.php file has been updated to include a config.json file in the same subdirectory. If the config.json exists, the contents will be used in place of any json configuration defined manually in the config.php file.

  • The new best practice is to use only config.json, not config.php any longer.

  • Since there is a new config.php file with this version, a new migration tool called upgrade_config.php has been included to easily install the new config.php file while still maintaining all of your existing settings.

    The upgrade_config.php command will create a new file called config.json containing all of the settings that used to be inside the config.php file.

    Important

    The upgrade_config.php tool should be NOT be RUN if the config.json file already exists!

    The upgrade_config.php tool should be RUN ONLY ONCE!

    COMPLETE ALL THE STANDARD UPGRADE STEPS FIRST!
    
    shell> cd {DASHBOARD_WEBROOT}/
    For example:
    shell> cd /volumes/data/www/tungsten/html/
    
    shell> ./upgrade_config.php `pwd`

    Important

    The upgrade_config.php tool should be NOT be RUN if the config.json file already exists!

    The upgrade_config.php tool should be RUN ONLY ONCE!

New! Cluster Tagging and Filtering Feature

  • You are now able to specify one or more tags per cluster in all cluster definition forms.

  • You are now able to filter the cluster display by a tag from the top search bar.

Support for both v6/APIv1 and v7/APIv2 clusters at the same time!

Support for per-cluster API User and Password!

  • You are now able to specify the API settings on a per-cluster basis (apiVersion, apiAuth, apiUser, apiPassword and apiSSL), allowing for mixed APIv1 and APIv2 clusters in the same Dashboard session, and clusters with different admin user/password pairs.

  • In previous versions of the Tungsten Dashboard, the API Version, API Authentication/User/Password and API Encryption settings were global only. This meant that a version 6 cluster running APIv1 and a version 7 cluster running APIv2 would not work at the same time in the same Dashboard sessioa, nor would clusters with different API user/password pairs.

  • The new configuration fields have been added to the display, add and edit cluster definition forms in all areas of the dashboard.

  • The API settings for each cluster are now displayed upon hover over the cluster service name. Each option will also show if the value is derived from the global default, or from a cluster-specific setting.

  • Expert mode now allows one-click toggling of global API authentication and SSL Encryption options in addition to the existing global API version toggle.

New Display Options for the REACT Frontend GUI

  • Two new settings have been added to the Tungsten Dashboard to support an improved GUI experience.

  • Both of the new settings are available via the GUI (config.php options in parens):

    • Display Style (displayStyle)

      Specify which horizontal display method to use in the new REACT frontend (Fill Space or Compressed)

    • Home Page (homepage)

      Specify which page to display first in the new REACT frontend (Cluster, Dashboard or Metrics)

New Cluster Rename Behavior

  • When changing a composite cluster parent name, the children are now updated with the new parent name. Previously, this was a manual operation.

  • when changing a composite cluster child name, the parent is now updated with the new child name. Previously, this was a manual operation.

New Tab Bar Behaviors for Improved Performance

  • A new setting (enableTabs) has been added to the Tungsten Dashboard to control the Tab bar behavior.

  • While getting the information to populate the Tab bar is a lightweight AJAX call, the actual data gathering in the backend is quite heavy, and requires an API call to every cluster. This backend processing can cause slowdowns in response time for the Dashboard and is why the new default is to have "dumb" Tabs and better performance.

  • Now, by default, the Tab bar will operate in "dumb" mode, and will NOT display the quantity of nodes that are not in automatic mode, nor the quantity of nodes that are in the not ready state. Also, the Tab refresh button will not appear either.

  • Enabling Tabs in the Settings Panel restores the original behavior of displaying the counts and the refresh button.

Additional New Features of Note

  • The Tungsten Dashboard now fully supports CAA clusters in v6!

  • Added the ability to close windows using the Escape key, enabled by default. Disable in the Settings panel.

  • Added a Copy To Clipboard button to all cluster configuration display windows.

  • Added a refresh button to the missing cluster display box for convenience - no longer need to refresh the entire page

  • In expert mode, there is a new heartbeat trigger button per cluster in old frontend

  • Cleaned up error_log calls to make the log file as quiet as possible, and added new setting (enableVerbose) and the aassociated verbose_log() function.

  • When RBAC is enabled, a logout link is now visible at the bottom center of the footer. Also improved footer messaging when RBAC is disabled and/or basic auth is disabled.

  • For both auto and manual cluster definition, if you create a Dashboard Service ID (service name) with hyphens, they will be converted to underscores upon save due to the way Javascript handles id's with hyphens.

  • When useHAProxy is enabled, the Manager Port will now be set to 8201 instead of 8091 to avoid port conflicts when installed directly on a cluster node.

  • The browser-specific date and time display for auto-refresh has been shortened to better fit on the navbar.

3.6. Tungsten Dashboard 1.0.10 GA (7 March 2022)

Version End of Life. 6 March 2023

Tungsten Dashboard provides a web-based UI for monitoring and managing Tungsten Clustering deployments.

Tungsten Dashboard v1.0.10 provides a number of new features, improvements and bugfixes.

Prometheus and Grafana Integration

  • Two new settings have been added to the Tungsten Dashboard to support Prometheus and Grafana integration.

  • Both of the new settings are available via the GUI (config.php options in parens):

    • Enable Prometheus Integration (enablePrometheus)

      Enable integration with Prometheus to display data in a new window.

    • Enable Grafana Integration (enableGrafana)

      Enable integration with Grafana to display graphs inside the Dashboard.

Configurable CURL Timeouts

  • Two new settings have been added to the Tungsten Dashboard to help compensate for slow environmemnts where API calls may take longer to complete.

  • Both of the new settings are available via the GUI (config.php options in parens):

    • CURL GET Timeout (curlTimeoutGET)

      The timeout used when curl connects to the Manager for a GET-specific API call, in seconds.

    • CURL POST Timeout (curlTimeoutPOST)

      The timeout used when curl connects to the Manager for a POST-specific API call, in seconds.

New Audit Trail Feature

  • The ability to track all write API calls made has been added.

  • There will be one audit file per day created in the auditDir (default: {WEBROOT}/audit.d).

  • One line per write containing: timestamp, ipaddr, user, role and msg/action

  • The new setting is partially available via the GUI (config.php options in parens):

    • Enable Audit Trail (enableAudit)

      Capture all POST API calls to a file in the {WEBROOT}/audit.d subdir like audit-{YYMMDD}.log

    • Audit subdirectory name (auditDir) (NOT available via GUI)

      The directory used to store the audit files (default: {WEBROOT}/audit.d)

New Notes-Per-Node Feature

  • You can now store text on a per-node basis.

  • There will be one note file per node created in the notesDir (default: {WEBROOT}/notes.d).

  • The new setting is partially available via the GUI (config.php options in parens):

    • Enable Per-Node Notes (enableNotes)

      Turn on the notes per node feature to capture text to a file in the {WEBROOT}/notes.d subdir, named like {SERVICE}-{FQDN}.txt

    • Note Icon (noteGlyphicon) (NOT available via GUI)

      Use noteGlyphicon to specify the note-per-node Glyphicon (default: comment).

    • Notes subdirectory name (notesDir) (NOT available via GUI)

      The directory used to store the note files (default: {WEBROOT}/notes.d)

New API URL Display Feature

  • You can now enable the display of the back-end API call URLs for transparency and learning.

  • The new setting is available via the GUI (config.php options in parens):

    • Enable API URL Display (enableURLDisplay)

      Display the API call URL for each command run.

New Flag-On-Lag Feature

  • You can now automatically highlight node rows where the Replicator is lagging by a specified number of seconds.

  • The new setting is available via the GUI (config.php options in parens):

    • Flag-On-Lag Delay (flagOnLagDelay) (in seconds)

      Set this option to a non-zero value to enable node row highlighting when the Replica is more than the specified number of seconds behind the Primary.

    • Flag-On-Lag Color (flagOnLagColor)

      The background color to use when marking a node row as too far behind. One of Info (blue), Warning (yellow) or Danger (red).

3.7. Tungsten Dashboard 1.0.9 GA (12 August 2020)

Version End of Life. 11 August 2021

Tungsten Dashboard provides a web-based UI for monitoring and managing Tungsten Clustering deployments.

Tungsten Dashboard v1.0.9 provides a number of new features, improvements and bugfixes.

Dashboard Configuration

  • Now able to configure Dashboard settings via the browser

    You can disable the editing of settings in the browser by changing the value of disableSettingsEdit to 1 in the config.php file, in the "settings": { } stanza:

     "disableSettingsEdit": 1 
  • All settings configured via the browser page are stored in the {webroot}/settings.d/ directory as individual JSON text files named for the setting. Please ensure it exists and is writable by the web server user.

  • You may edit or delete any of the files in the {webroot}/settings.d/ directory. The setting will revert to the default if deleted. you may also choose to configure settings in this way as opposed to using the config.php file. Your choice.

  • Refactored all options and created centralized defaults

Software Update

  • Now able to self-update the Dashboard software via the browser

    There are four related settings, enableUpdates, tmpDir, downloadAccessKey and downloadSecretKey.

    All four must be located in the config.php file, in the "settings": { } stanza. They are not accessible from the browser settings page.

    You can disable the Dashboard self-update feature by changing the value of enableUpdates to 0 in config.php (default: 1):

     "enableUpdates": 1 

    The tmpDir value is used to determine where downloaded software packages are saved to:

     "tmpDir":"/tmp" 

    The other two (downloadAccessKey and downloadSecretKey) need to be obtained from Continuent support and typicially ship with the Dashboard installation package.

Cluster Definitions

  • Now able to manually create and save cluster definitions in the conf.d subdirectory. Originally, a cluster could only be defined in the "clusters": { } stanza.

  • Now able to create and save cluster definitions to the conf.d subdirectory via a browser workflow

  • Added Display, Edit and Remove Cluster Definition menu choices for each cluster

  • Now able to automatically define cluster definitions in conf.d just by providing a hostname and port number in a browser workflow

  • Now able to automatically define cluster definitions in conf.d at Dashboard startup

    There are three related settings, enableAutoConfiguration, managerPort and useHAProxy.

    You can enable the Dashboard auto-configuration feature by changing the "Enable Auto-Configuration?" setting via the Dashboard settings page in the browser, or changing the value of enableAutoConfiguration to 1 in config.php (default: 0) or via the Dashboard settings page in the browser:

     "enableAutoConfiguration": 1 

    The managerPort value is used to determine what port to communicate with the manager upon when performing auto-configuration and auto-define, as well as populating form fields in other places. Only change this if you have change the API listener port for the Manager as well.

     "managerPort": 8090 

    The useHAProxy value is used to determine how to calculate ports when performing auto-configuration and auto-define.

    Set the value to 1 to determine the manager port number automatically during various operations based on calulations using the base managerPort.

    Set the value to 0 (default) to use the base managerPort with no attempt to auto-define the port.

    You can enable the manager port auto-configuration feature by changing the "Using HA Proxy?" setting via the Dashboard settings page in the browser, or changing the value in the config.php file.

     "useHAProxy": 1 

UI/UX

  • Role name cleaning (Master is now Primary, and Slave is now Replica for nodes; Master is now Active, and Slave is now Passive for clusters)

  • Improve error handling for JSON responses to AJAX calls

  • Bug fixes in service alias support

  • Many footer improvements, including a link to check for an available Dashboard software update

  • Stop providing tabInfo during intitial page load, instead do it as AJAX call after load to save initial page load time

Dashboard Diagnostics

  • Now able to upload a Dashboard Diagnostic containing the JSON configuration to Continuent Support's protected AWS bucket. No other customer has access to this location, it is upload-only.

    There are three related settings, customerName, uploadAccessKey and uploadSecretKey.

    The customerName value is used to pre-populate the diagnostic upload form.

     "customerName":"your customer name here" 

    The other two (uploadAccessKey and uploadSecretKey) need to be located in config.php

     "uploadAccessKey":"AKIAIWDZPQUE5YL4SBDQ", ] 
     "uploadSecretKey":"FQ0iVkTtH9biIZT2+IpwXwhqXvVwqMUqsZ4++N4K" ] 

Misc Admin

  • New Expert mode disables both confirmation prompts when Deleting All Definitions

    The default is 0 (disabled). Set enableExpertMode to 1 (one) to enable.

     "enableExpertMode": 1 
  • Use the enableDebug setting to get additional logging information and use the debug software versions when checking for an available update.

     "enableDebug": 1 

3.8. Tungsten Dashboard 1.0.8 GA (4 June 2020)

Version End of Life. 3 June 2021

Tungsten Dashboard provides a web-based UI for monitoring and managing Tungsten Clustering deployments.

Tungsten Dashboard v1.0.8 provides a number of new features, improvements and bugfixes.

  • Added basic Role-Based Access Control (RBAC). There are two roles, Administrator with full access and Operator with Read-Only access. This feature requires Basic Auth to be properly configured on the Web server.

    When enabled, the user's current role will be displayed in the footer. Refresh the page to activate any changes to config.php.

    The default is 0 (disabled). Set enableRBAC to 1 (one) to enable.

     "enableRBAC":1 

    Use the administrators setting to list the users with admin privs:

     "administrators": [ "adminUser1","adminUser2" ] 
  • Improved page load performance via caching of API calls. This is especially helpful with Composite clusters that have multiple sites over a wide area.

  • Added the ability to modify the browser window title using the new configuration option windowTitle

  • Added the ability to change the cluster service sort order from the alpha default to as-written configuration order using the new configuration option sortByConfigOrderNotAlpha

  • Site favicons along with the navigation bar logo and colors have been updated to promote a cleaner look. Additional icon replacements and color tweaks have been made throughout the tool.

  • Added hover-based tooltips for all fields and buttons where possible. Set disableTooltips to 1 to prevent the tooltips from appearing.

  • Significantly improved the Connector popover formatting, sorting and operation.

  • Message handling is improved so that multiple actions and responses are tracked and messaged properly.

  • Added the ability to view the json configuration in the browser via a menu link.

  • Added the ability to check for Dashboard software updates.

  • Added the ability to check for Clustering software updates on a per-node basis.

Tungsten Dashboard is compatible with both the Tungsten Clustering 5.3.x series and 6.x series.

3.9. Tungsten Dashboard 1.0.7 GA (26 November 2019)

Version End of Life. 26 November 2020

Tungsten Dashboard provides a web-based UI for monitoring and managing Tungsten Clustering deployments.

Tungsten Dashboard v1.0.7 provides a number of new features, improvements and bugfixes.

  • Added the feature to allow for cluster service name aliases. You may now add the sub-key actualName pointing to the "real" name of the service, and change the top-level cluster service name to some alias that you understand.

    Previously, it was impossible to configure two or more clusters with the same service name. This could be required if clusters were installed into different environments like production, staging or development. While the best practice is to name the cluster services to match the environment (i.e. east_prod and east_staging), in some situations this may not be possible.

  • Added a new feature to automatically fade out messages after a delay. The default is 60 seconds. Set msgFadeOutTimer to 0 (zero) to disable or to a positive integer to specify the delay in seconds.

     "msgFadeOutTimer":60 
  • Improved the look & feel of the overall layout, including display widths, the location of the timestamp marker and spacing.

  • Fixed a bug where the controls to open and close a cluster were STILL not working.

  • Fixed a bug where the datasource status details hover was not displaying properly

Tungsten Dashboard is compatible with both the Tungsten Clustering 5.3.x series and 6.x series.

3.10. Tungsten Dashboard 1.0.6 GA (3 September 2019)

Version End of Life. 3 September 2020

Tungsten Dashboard provides a web-based UI for monitoring and managing Tungsten Clustering deployments.

Tungsten Dashboard v1.0.6 is a bugfix and minor feature release.

  • Fixed a bug where the controls to open and close a cluster were not working.

  • When Auto-refresh is turned on, any issuence of a command will stop the auto-refresh. Simply re-select your desired refresh rate to turn it back on.

Tungsten Dashboard is compatible with both the Tungsten Clustering 5.3.x series and 6.x series.

3.11. Tungsten Dashboard 1.0.5 GA (28 June 2019)

Version End of Life. 28 June 2020

Tungsten Dashboard provides a web-based UI for monitoring and managing Tungsten Clustering deployments.

Tungsten Dashboard v1.0.5 is a bugfix release.

  • Fixed CMM cluster bug where clusters other than the first do not show subservices.

  • Tweaked cell alignment

Tungsten Dashboard is compatible with both the Tungsten Clustering 5.3.x series and 6.x series.

3.12. Tungsten Dashboard 1.0.4 GA (11 April 2019)

Version End of Life. 11 April 2020

Tungsten Dashboard provides a web-based UI for monitoring and managing Tungsten Clustering deployments.

Tungsten Dashboard v1.0.4 is a bugfix release.

  • Fixed cluster-level open/close regression.

  • Tweaked error text and reduced noise in the logs.

Tungsten Dashboard is compatible with both the Tungsten Clustering 5.3.x series and 6.x series.

3.13. Tungsten Dashboard 1.0.3 GA (22 March 2019)

Version End of Life. 22 March 2020

Tungsten Dashboard provides a web-based UI for monitoring and managing Tungsten Clustering deployments.

Tungsten Dashboard v1.0.3 is a feature release for better global controls and customization.

The default for navButtonFormat is icon if not specified.

  • Added modal "Stop Auto-Refresh" button which will turn off the Auto-refresh feature. This button is only visible if auto-refresh is enabled.

  • Added ability to set global buttons to icon, text or some combination. Use the setting navButtonFormat and specify one or more of icon or text as a comma-separated string, no spaces. Order counts.

    $jsonConfig = <<<EOJ
    {
     "settings": {
     "navButtonFormat":"icon",
    ...
    EOJ;

    Currently there are four (4) possible entries:

    "navButtonFormat":"icon",
    	"navButtonFormat":"text",
    	"navButtonFormat":"icon,text",
    	"navButtonFormat":"text,icon",

Tungsten Dashboard is compatible with both the Tungsten Clustering 5.3.x series and 6.x series.

3.14. Tungsten Dashboard 1.0.2 GA (20 September 2018)

Version End of Life. 20 September 2019

Tungsten Dashboard provides a web-based UI for monitoring and managing Tungsten Clustering deployments.

Tungsten Dashboard v1.0.2 is a bug fix release for better API error handling.

  • Refactored API calls for better error handling.

  • Better error reporting on the front-end.

Tungsten Dashboard is compatible with both the Tungsten Clustering 5.3.x series and 6.x series.

3.15. Tungsten Dashboard 1.0.1 GA (17 September 2018)

Version End of Life. 17 September 2019

Tungsten Dashboard provides a web-based UI for monitoring and managing Tungsten Clustering deployments.

Tungsten Dashboard v1.0.1 is a bug fix release that also contains a few improvements.

  • Support for Composite Active/Active topology offered in Continuent Clustering v6.x (requires Continuent Clustering version 6.0.3)

  • Improvements to the menu system layout and clarity

  • Composite-level cluster commands have been relocated to a new menu to the right of the State field

  • Composite clusters now display the actual composite state instead of the Ready/Warning/Error status indicators, and status indicator lights have been moved to the left of the State label

  • Improvements to the locking system:

    • Auto-Lock and Auto-Unlock are now both configurable via config.php

    • Auto-Lock and Auto-Unlock setting are now both visible at the bottom of the cluster-level locking menu

    • Auto-Lock may be configured to attempt a lock for all actions, heartbeats only, or not at all

    • Auto-Unlock may be configured to attempt an unlock for all actions, heartbeats only, or not at all

  • Additional formatting tweaks, including the reduction in height of the rows

Tungsten Dashboard is compatible with both the Tungsten Clustering 5.3.x series and 6.x series.

3.16. Tungsten Dashboard 1.0.0 GA (10 May 2018)

Version End of Life. 10 May 2019

Tungsten Dashboard provides a web-based UI for monitoring and managing Tungsten Clustering deployments.

It supports the following features:

  • Full monitoring information on the status and progress of replication and the status of the cluster

  • Monitor multiple clusters through a single page

  • Perform switches and failovers

  • Shun hosts

  • Recover failed hosts

Tungsten Dashboard is compatible with the Tungsten Clustering 5.3.x series.

4. Tungsten AMI Release Notes

4.1. Tungsten AMI 5.0.0 GA (NYR)

Version End of Life. Not Yet Set

This AMI release resolves Amazon Linux OS vulnerabilities identified by AWS and a number of small improvements in the deployment Wizard for manual configurations.

Bug Fixes

  • Operating System/3rd Party Packages

    • Addresses OS security vulnerability identifed by AWS (More info at https://alas.aws.amazon.com/AL2/ALAS-2019-1367.html)

    • Deployment Wizard now correctly configures the tungsten.ini for the MongoDB Atlas applier.

      Issues: CT-1161

    • When configuring a cluster-extractor to apply into AWS Redshift, the s3.json configuration fiule would be named incorrectly.

      Issues: CT-1219

4.2. Tungsten AMI 4.0.0 GA (17 December 2020)

Version End of Life. Not Yet Set

This AMI release brings the Tungsten Replicator and Tungsten Clustering AMI's in line with the latest release of the core products, in addition new Cloudformation Templates where released to allow easier deployment of Tungsten Clustering.

Improvements, new features and functionality

  • Installation and Deployment

    • The MongoDB applier now supports replication into MongoDB Atlas.

Bug Fixes

4.3. Tungsten AMI 3.0.0 GA (20 July 2020)

Version End of Life. 17 December 2020

The v3.0 AMI release introduces two new AMI's for Tungsten Clustering.

Known Issues

  • Installation and Deployment

    • Cloudformation support for Composite Active/Active and Composite Active/Passive clusters will only allow all clusters to be deployed within the same AWS region. This will be addressed in a future release.

4.4. Tungsten AMI 2.0.0 GA (11 November 2019)

Version End of Life. 17 December 2020

The v2.0 AMI release includes additional Targets and a more simplified deployment wizard.

Improvements, new features and functionality

  • Behavior Changes

    • Additional support added for the following Targets:

      • Oracle

      • Cassandra

      • Clickhouse

      • Elasticsearch

      • Hadoop

      • MongoDB

    • Support added to allow easy configuration of extraction from, or replication into, an existing Tungsten Cluster.

    • Installation Wizard now makes it easier to enable additional properties, such as SSL and basic filtering

    • Ships with the latest v6.1.1 release of Tungsten Replicator

4.5. Tungsten AMI 1.0.0 GA (6 June 2019)

Version End of Life. 11 November 2019

The first release of the Tungsten AMI on the Amazon Marketplace is available and supports replication out of MySQL (Community Edition, Enterprise Edition, Percona, MariaDB, Amazon RDS/Aurora) into the following targets:

  • MySQL (Community Edition, Enterprise Edition, Percona, MariaDB, Amazon RDS/Aurora)

  • AWS Redshift

  • Vertica

  • PostgreSQL

  • Kafka

Known Issues

  • Installation and Deployment

    • Additional Targets will be supported in a future release.