This filter was introduced in version 6.1.12
Maps charsets in newer MySQL 8 environments allowing THL to be applied to older versions of MySQL.
| Pre-configured filter name |
mapcharset
| ||
| Classname |
com.continuent.tungsten.replicator.filter.mapcharset
| ||
| Property prefix |
replicator.filter.mapcharset
| ||
| Stage compatibility | q-to-dbms | ||
| tpm Option compatibility | --svc-applier-filters | ||
| Data compatibility | Any event | ||
| Parameters | |||
| (none) | |||
The filter must not be used if applying into MySQL 8 or greater.
Once a replica has been upgraded to MySQL 8 or greater, the filter MUST be disabled.
The filter utilisies the tungsten-replicator/support/filters-config/mapcharset.json to determine the source/target charset mappings.
The json file is a very simple array structure in the format of <SourceCharset>: <TargetCharset>
The default installation of this filter should cover most, if not all, scenarios.
This, however, can be changed by updating mapcharset.json file which maps MySQL collation id, as seen by the following mysql query example:
mysql> select * from INFORMATION_SCHEMA.COLLATIONS where character_set_name like 'utf8mb4' order by id;
+----------------------------+--------------------+-----+------------+-------------+---------+---------------+
| COLLATION_NAME | CHARACTER_SET_NAME | ID | IS_DEFAULT | IS_COMPILED | SORTLEN | PAD_ATTRIBUTE |
+----------------------------+--------------------+-----+------------+-------------+---------+---------------+
| utf8mb4_general_ci | utf8mb4 | 45 | | Yes | 1 | PAD SPACE |
| utf8mb4_bin | utf8mb4 | 46 | | Yes | 1 | PAD SPACE |
| utf8mb4_unicode_ci | utf8mb4 | 224 | | Yes | 8 | PAD SPACE |
| utf8mb4_icelandic_ci | utf8mb4 | 225 | | Yes | 8 | PAD SPACE |
...
| utf8mb4_0900_ai_ci | utf8mb4 | 255 | Yes | Yes | 0 | NO PAD |
| utf8mb4_de_pb_0900_ai_ci | utf8mb4 | 256 | | Yes | 0 | NO PAD |
| utf8mb4_is_0900_ai_ci | utf8mb4 | 257 | | Yes | 0 | NO PAD |
| utf8mb4_lv_0900_ai_ci | utf8mb4 | 258 | | Yes | 0 | NO PAD |
| utf8mb4_ro_0900_ai_ci | utf8mb4 | 259 | | Yes | 0 | NO PAD |
...
| utf8mb4_ru_0900_as_cs | utf8mb4 | 307 | | Yes | 0 | NO PAD |
| utf8mb4_zh_0900_as_cs | utf8mb4 | 308 | | Yes | 0 | NO PAD |
| utf8mb4_0900_bin | utf8mb4 | 309 | | Yes | 1 | NO PAD |
+----------------------------+--------------------+-----+------------+-------------+---------+---------------+
75 rows in set (0.00 sec)
For example, the default mapping contains :
{
...
"255": "45"
...
}
This means that collation utf8mb4_0900_ai_ci is mapped to utf8mb4_general_ci.
tungsten-replicator/support/filters-config/mapcharset.readme contains a full list of the default mappings.