ColumnName Filter
The ColumnNameFilter loads the table specification information for tables and adds this information to the THL data for information
extracted using row-based replication.
| Pre-configured filter name | colnames |
| Classname | com.continuent.tungsten.replicator.filter.ColumnNameFilter |
| Property prefix | replicator.filter.colnames |
| Stage compatibility | binlog-to-q |
tpm Option compatibility | svc-extractor-filters |
| Data compatibility | Row events |
| Keeps Cached Data | Yes |
| Cached Refreshed When? | Emptied when going OFFLINE; Updated when ALTER statement seen |
| Metadata Updated | Yes |
Parameter
| Parameter | Type | Default | Description |
|---|---|---|---|
user | string | ${replicator.global.extract.db.user} | The username for the connection to the database for looking up column definitions |
password | string | ${replicator.global.extract.db.password} | The password for the connection to the database for looking up column definitions |
url | string | jdbc:mysql:thin://${replicator.global.extract.db.host}:${replicator.global.extract.db.port}/${replicator.schema}?createDB=true | JDBC URL of the database connection to use for looking up column definitions |
addSignedFlag | boolean | true | Determines whether the signed flag information for columns should be added to the metadata for each column. |
ignoreMissingTables | boolean | true | When true, tables that do not exist will not trigger metadata and column names to be added to the THL data. |
This filter is designed to be used for testing and with heterogeneous replication where the field name information can be used to construct and build target data structures.
The filter is required for the correct operation of heterogeneous replication, for example when replicating to MongoDB. The filter works by using the replicator username and password to access the underlying database and obtain the table definitions. The table definition information is cached within the replication during operation to improve performance.
When extracting data from the binary log using row-based replication, the column names for each row of changed data are added to the THL.
Enabling this filter changes the THL data from the following example, shown without the column names:
SEQ# = 27 / FRAG# = 0 (last frag)
- TIME = 2013-08-01 18:29:38.0
- EPOCH# = 11
- EVENTID = mysql-bin.000012:0000000000004369;0
- SOURCEID = host31
- METADATA = [mysql_server_id=1;dbms_type=mysql;service=alpha;shard=test]
- TYPE = com.continuent.tungsten.replicator.event.ReplDBMSEvent
- OPTIONS = [foreign_key_checks = 1, unique_checks = 1]
- SQL(0) =
- ACTION = INSERT
- SCHEMA = test
- TABLE = sales
- ROW# = 0
- COL(1: ) = 1
- COL(2: ) = 23
- COL(3: ) = 45
- COL(4: ) = 45000.00
To a version where the column names are included as part of the THL record:
SEQ# = 43 / FRAG# = 0 (last frag)
- TIME = 2013-08-01 18:34:18.0
- EPOCH# = 28
- EVENTID = mysql-bin.000012:0000000000006814;0
- SOURCEID = host31
- METADATA = [mysql_server_id=1;dbms_type=mysql;service=alpha;shard=test]
- TYPE = com.continuent.tungsten.replicator.event.ReplDBMSEvent
- OPTIONS = [foreign_key_checks = 1, unique_checks = 1]
- SQL(0) =
- ACTION = INSERT
- SCHEMA = test
- TABLE = sales
- ROW# = 0
- COL(1: id) = 2
- COL(2: country) = 23
- COL(3: city) = 45
- COL(4: value) = 45000.00
When the row-based data is applied to a non-MySQL database, the column name information is used by the applier to specify the column, or the key when the column and value is used as a key/value pair in a document-based store.