Version End of Life. 15 Aug 2024
Release 6.1.5 is a small interim bug fix with a number of issues resolved within the Core Replicator, sepcifically for heterogeneous environments.
The following changes have been made to Tungsten Replicator and may affect existing scripts and integration tools. Any scripts or environment which make use of these tools should check and update for the new configuration:
When using the batch applier in Heterogeneous pipelines (Redshift, Vertica, Hadoop) the batch applier removes DDL statements.
There may be occasions when you intentionally want DDL to pass through, such as if you have a custom filter that injects custom DDL statements into the pipeline, however the batch applier would always remove them.
A new property is now available to control this behaviour. Set
property=replicator.applier.dbms.applyStatements=trueto allow the batch applier to retain DDL statements. The default value offalseretains the original behaviour of removing DDL.Issues: CT-1270
The following issues are known within this release but not considered critical, nor impact the operation of Tungsten Replicator. They will be addressed in a subsequent patch release.
In MySQL release 8.0.21 the behavior of
CREATE TABLE ... AS SELECT ...has changed, resulting in the transactions being logged differenly in the binary log. This change in behavior will cause the replicators to fail.Until a fix is implemented within the replicator, the workaround for this will be to split the action into separate
CREATE TABLE ...followed byINSERT INTO ... SELECT FROM...statements.If this is not possible, then you will need to manually create the table on all nodes, and then skip the resulting error in the replicator, allowing the subsequent loading of the data to continue.
Issues: CT-1301
When replicating data that included timestamps, the replicator
would update the timestamp value to the time within the commit
from the incoming THL. When using statement based replication
times would be correctly replicated, but if using a mixture of
statement and row based replication, the timestamp value would
not be set back to the default time when switching between
statement and row based events. This would not cause problems in
the applied host, except when log_slave_updates
was enabled. In this case, all row-based events after a
statement based event would have the same timestamp value
applied.
This was most commonly seen when using the standalone replicator to replicate into a Cluster, either from a standlone source, or a cluster source.
Issues: CT-1255
For offboard extraction, the replicator would appear to be ONLINE but not
actually processing new events.
This is due to the relay client getting an incomplete packet from the remote database and going into a WAITING state.
To handle these situations, a new property has been included that will set a timeout and if the replicator
does not process an event in the given timeout period, we assume we have lost the link to the remote database and
will place the replicator into an OFFLINE:ERROR state.
Providing auto-recovery has been enabled using the auto-recovery-max-attempts parameter,
the replicator will then restart and proceed successfully.
The new property to include is property=replicator.extractor.dbms.relayLogIdleTimeout
The default value (0) will disable the timeout. Values provided are in seconds, so 300 would be 5 minutes.
Setting the timeout too low in quieter systems may result in unnecessary replicator restarts. The value should be set according to the activity levels of your database. If the source is very active with constant updates, then a low value would be appropriate. On quieter systems that may have long periods of inactivty, should have a timeout value set no less than the longest, normal, period of inactivity within your system.
Issues: CT-1262
If filtering is in use, and a space appeared either side of the delimiter in a "schema.table" definition in your SQL, the replicator would fail to parse the statement correctly.
For example, all of the examples below are valid SQL but would cause a failure in the replicator:
sql> CREATE TABLE myschema. mytable (.... sql> CREATE TABLE myschema .mytable (.... sql> CREATE TABLE myschema . mytable (....
Issues: CT-1278
Fixes a bug in the Drizzle Driver whereby a failing prepared statement that exceeds 1000 characters would report a
String index out of range: 999 error rather than the actual error.
Issues: CT-1303