The rowaddtxninfo filter examines an
entire row-based event within the THL and then builds a list of
transaction information which is embedded into the metadata for the event.
This can be combined with functionality in corresponding appliers, such as
Section 4.4, “Deploying the Kafka Applier” which can then be used and embedded
into messages or document-based databases.
| Pre-configured filter name |
rowaddtxninfo
| ||
| Classname |
com.continuent.tungsten.replicator.filter.JavaScriptFilter
| ||
| Property prefix |
replicator.filter.rowaddtxninfo
| ||
| Stage compatibility | Any | ||
| tpm Option compatibility | |||
| Data compatibility | Row-events only | ||
| Parameters | |||
| Parameter | Type | Default | Description |
The rowaddtxninfo filter processes
an entire transaction to determine the following information:
List of schemas and tables affected by the transaction.
A count of the rows inserted, updated, or deleted per schema/table combination.
A total count of all the rows inserted, updated, or deleted across the entire transaction.
Once the information about the transaction has been collected, the data is then formulated into key/value pairs that are incorporated into the metadata for the entire THL event.
For example, when inserting information into the same table across three separate statements the but the same overall transaction, the THL with the filter enabled looks like the example below:
shell> thl list -last
SEQ# = 162 / FRAG# = 0 (last frag)
- TIME = 2018-03-02 08:47:14.0
- EPOCH# = 162
- EVENTID = mysql-bin.000065:0000000000000534;-1
- SOURCEID = trfiltera
- METADATA = [mysql_server_id=366;dbms_type=mysql;tz_aware=true;strings=utf8;service=alpha;shard=msg;
» tungsten_filter_columnname=true; tungsten_filter_primarykey=true;tungsten_filter_enumtostring=true;
» txinfo_rowcount_msg.msg=1;txinfo_rowcount_msg.msgsub=2;txinfo_rowcount=3]
- TYPE = com.continuent.tungsten.replicator.event.ReplDBMSEvent
- OPTIONS = [foreign_key_checks = 1, unique_checks = 1, time_zone = '+00:00']
- SQL(0) =
- ACTION = INSERT
- SCHEMA = msg
- TABLE = msg
- ROW# = 0
- COL(1: id) = 108
- COL(2: msg) = txinfo
- COL(3: msg2) = txinfo
- KEY(1: id) = NULL
- OPTIONS = [foreign_key_checks = 1, unique_checks = 1, time_zone = '+00:00']
- SQL(1) =
- ACTION = INSERT
- SCHEMA = msg
- TABLE = msgsub
- ROW# = 0
- COL(1: id) = 108
- COL(2: msg) = subtx
- ROW# = 1
- COL(1: id) = 108
- COL(2: msg) = subxtExamining the metadata more closely you can see the transaction information:
... txinfo_rowcount_msg.msg=1;txinfo_rowcount_msg.msgsub=2;txinfo_rowcount=3 ...
This can be further extrapolated as:
txinfo_rowcount_msg.msg=1
One row has been updated in the
msg.msg schema/table.
txinfo_rowcount_msg.msgsub=2
Two rows has been updated in the
msg.submsg schema/table.
txinfo_rowcount=3
A total of three rows have been updated within the transaction.
If you transactional information is needed within a Kafka deployment, this filter must have been enabled for the transaction information to be included. See Section 4.4.2.1, “Optional Configuration Parameters for Kafka”, for more information.