Continuent Documentation

Search:

10.4.14. dropcolumn.js Filter

The dropcolumn filter enables columns in the THL to be dropped. This can be useful when replicating Personal Identification Information, such as email addresses, phone number, personal identification numbers and others are within the THL but need to be filtered out on the slave.

Pre-configured filter name dropcolumn
JavaScript Filter File tungsten-replicator/samples/extensions/javascript/dropcolumn.js
Property prefix replicator.filter.dropcolumn
Stage compatibility binlog-to-q, q-to-dbms
tpm Option compatibility --svc-extractor-filters, --svc-applier-filters
Data compatibility Any event
Parameters
Parameter Type Default Description
definitionsFile Filename ~/dropcolumn.json Location of the definitions file for dropping columns

The filter is available by default as dropcolumn, and the filter is configured through a JSON file that defines the list of columns to be dropped. The filter relies on the colnames filter being enabled.

To enable the filter:

shell> tpm update --svc-extractor-filters=colnames,dropcolumn \
    --property=replicator.filter.dropcolumn.definitionsFile=/opt/continuent/share/dropcolumn.json

A sample configuration file is provided in /opt/continuent/share/dropcolumn.json. The format of the file is a JSON array of schema/table/column specifications:

[
    {
        "schema": "vip",
        "table": "clients",
        "columns": [
            "personal_code",
            "birth_date",
            "email"
        ]
    },
    ...
]

Where:

  • schema specifies the name of the schema on which to apply the filtering. If * is given, all schemas are matched.

  • table specifies the name of the table on which to apply the filtering. If * is given, all tables are matched.

  • columns is an array of column names to be matched.

For example:

[
    {
        "schema": "vip",
        "table": "clients",
        "columns": [
            "personal_code",
            "birth_date",
            "email"
        ]
    },
    ...
]

Filters the columns email, birth_date, and personal_code within the clients table in the vip schema.

To filter the telephone column in any table and any schema:

[
    {
        "schema": "*",
        "table": "*",
        "columns": [
            "telephone"
        ]
    }
]

Care should be taken when dropping columns on the slave and master when the column order is different or when the names of the column differ: