rowaddtxninfo filter examines an
entire row-based event within the THL and then builds a list of
transaction information which is embedded into the metadata for the event.
This can be combined with functionality in corresponding appliers, such as
Deploying MySQL to Kafka Replication which can then us and embedded
into messages or document-based databases.
|Pre-configured filter name||
|tpm Option compatibility|
|Data compatibility||Row-events only|
rowaddtxninfo filter processes
an entire transaction to determine the following information:
List of schemas and tables affected by the transaction.
A count of the rows inserted, updated, or deleted per schema/table combination.
A total count of all the rows inserted, updated, or deleted across the entire transaction.
Once the information about the transaction has been collected, the data is then formulated into key/value pairs that are incorporated into the metadata for the entire THL event.
For example, when inserting information into the same table across three separate statements the but the same overall transaction, the THL with the filter enabled looks like the example below:
thl list -lastSEQ# = 162 / FRAG# = 0 (last frag) - TIME = 2018-03-02 08:47:14.0 - EPOCH# = 162 - EVENTID = mysql-bin.000065:0000000000000534;-1 - SOURCEID = trfiltera - METADATA = [mysql_server_id=366;dbms_type=mysql;tz_aware=true;strings=utf8;service=alpha;shard=msg;tungsten_filter_columnname=true; » tungsten_filter_primarykey=true;tungsten_filter_enumtostring=true; » txinfo_rowcount_msg.msg=1;txinfo_rowcount_msg.msgsub=2;txinfo_rowcount=3] - TYPE = com.continuent.tungsten.replicator.event.ReplDBMSEvent - OPTIONS = [foreign_key_checks = 1, unique_checks = 1, time_zone = '+00:00'] - SQL(0) = - ACTION = INSERT - SCHEMA = msg - TABLE = msg - ROW# = 0 - COL(1: id) = 108 - COL(2: msg) = txinfo - COL(3: msg2) = txinfo - KEY(1: id) = NULL - OPTIONS = [foreign_key_checks = 1, unique_checks = 1, time_zone = '+00:00'] - SQL(1) = - ACTION = INSERT - SCHEMA = msg - TABLE = msgsub - ROW# = 0 - COL(1: id) = 108 - COL(2: msg) = subtx - ROW# = 1 - COL(1: id) = 108 - COL(2: msg) = subxt
Examining the metadata more closely you can see the transaction information:
... txinfo_rowcount_msg.msg=1;txinfo_rowcount_msg.msgsub=2;txinfo_rowcount=3 ...
This can be further extrapolated as:
One row has been updated in the
Two rows has been updated in the
A total of three rows have been updated within the transaction.
If you transactional information is needed within a Kafka deployment, this filter must have been enabled for the transaction information to be included. See Optional Configuration Parameters for Kafka, for more information.