binarystringconversion Filter

Version availability

Available in 7.1.2 and later.

Provides a way to convert varchar data from one charset into another.

This filter is particularly beneficial when used with heterogeneous replication enabling safe transformation of MySQL varchar fields.

Pre-configured filter name	binarystringconversionfilter
Classname	com.continuent.tungsten.replicator.filter.binarystringconversion
Property prefix	replicator.filter.binarystringconversion
Stage compatibility
tpm Option compatibility	svc-applier-filters
Data compatibility	ROW data

Parameters

Parameter	Type	Default	Description
definitionsFile	Filename	~/binarystringconversion.json	Location of the definitions file defining rules for conversion

The filter utilizes the tungsten-replicator/support/filters-config/binarystringconversion.json to determine the rules for conversion, an example of this file is below:

The JSON file details the charset to be used for specific columns and/or entire tables, as well as the ability to define a default for all rows in a table/schema.

The wildcard (*) can be used to signal that the rules apply to all tables and/or all schemas

[
 {
    "columns" : [
       {"name":"id", "charset":"utf8"},
       {"name":"company_id", "charset":"cp1252"}
    ],
    "table" : "accounts",
    "schema" : "*"
 },
 {
    "columns" : [
       {"name":"id", "charset":"cp1252"}
    ],
    "table" : "addresses",
    "schema" : "*",
    "default" : "UTF8"
 },
 {
    "columns" : [
       {"name":"jap_id", "charset":"ISO-10646-UCS-2"}
    ],
    "table" : "*",
    "schema" : "*",
    "default" : "cp1252"
 },
 {
    "columns" : [
       {"name":"id", "charset":"utf8"},
       {"name":"company_id", "charset":"cp1252"},
       {"name":"jap_id", "charset":"ISO-10646-UCS-2"},
    ],
    "table" : "accounts2",
    "schema" : "test3"
 }
]