The rename filter enables schemas to
be renamed at the database, table and column levels, and for complex
combinations of these renaming operations. Configuration is through a CSV
file that defines the rename parameters. A single CSV file can contain
multiple rename definitions. The rename operations occur only on ROW based
events.
The rename filter also performs
schema renames on statement data.
| Pre-configured filter name |
rename
| ||
| Classname |
com.continuent.tungsten.replicator.filter.RenameFilter
| ||
| Property prefix |
replicator.filter.rename
| ||
| Stage compatibility | |||
| tpm Option compatibility | |||
| Data compatibility | Row events. | ||
| Parameters | |||
| Parameter | Type | Default | Description |
definitionsFile
| string |
{replicator.home.dir}/samples/extensions/java/rename.csv
| Location of the CSV file that contains the rename definitions. |
The CSV file is only read when an explicit reconfigure operation is triggered. If the file is changed, a configure operation (using tpm update) must be initiated to force reconfiguration.
To enable using the default CSV file:
shell> ./tools/tpm update alpha --svc-applier-filters=rename
The CSV consists of multiple lines, one line for each rename
specification. Comments are supposed using the
# character.
The format of each line of the CSV is:
originalSchema,originalTable,originalColumn,newSchema,newTable,newColumn
Where:
originalSchema,
originalTable,
originalColumn define the
original schema, table and column.
Definition can either be:
Explicit schema, table or column name
* character, which indicates
that all entries should match.
newSchema,
newTable,
newColumn define the new schema,
table and column for the corresponding original specification.
Definition can either be:
Explicit schema, table or column name
- character, which indicates
that the corresponding object should not be updated.
For example, the specification:
*,chicago,*,-,newyork,-
Would rename the table chicago in
every database schema to newyork.
The schema and column names are not modified.
The specification:
*,chicago,destination,-,-,source
Would match all schemas, but update the column
destination in the table
chicago to the column name
source, without changing the schema
or table name.
Processing of the individual rules is executed in a specific order to allow for complex matching and application of the rename changes.
Rules are case sensitive.
Schema names are looked up in the following order:
schema.table (explicit
schema/table)
schema.* (explicit schema,
wildcard table)
Table names are looked up in the following order:
schema.table (explicit
schema/table)
*.table (wildcard schema,
explicit table)
Column names are looked up in the following order:
schema.table (explicit
schema/table)
schema.* (explicit schema,
wildcard table)
*.table (wildcard schema,
explicit table)
*.* (wildcard schema,
wildcard table)
Rename operations match the first specification according to the above rules, and only one matching rule is executed.
When processing multiple entries that would match the same definition, the above ordering rules are applied. For example, the definition:
asia,*,*,america,-,- asia,shanghai,*,europe,-,-
Would rename asia.shanghai to
europe.shanghai, while renaming
all other tables in the schema
asia to the schema
america. This is because the
explicit schema.table rule is
matched first and then executed.
Complex renames involving multiple schemas, tables and columns can be achieved by writing multiple rules into the same CSV file. For example given a schema where all the tables currently reside in a single schema, but must be renamed to specific continents, or to a 'miscellaneous' schema, while also updating the column names to be more neutral would require a detailed rename definition.
Existing tables are in the schema
sales:
chicago newyork london paris munich moscow tokyo shanghai sydney
Need to be renamed to:
northamerica.chicago northamerica.newyork europe.london europe.paris europe.munich misc.moscow asiapac.tokyo asiapac.shanghai misc.sydney
Meanwhile, the table definition needs to be updated to support more complex structure:
id area country city value type
The area is being updated to contain the region within the country,
while the value should be renamed to the three-letter currency code, for
example, the london table would
rename the value column to
gbp.
The definition can be divided up into simple definitions at each object level, relying on the processing order to handle the individual exceptions. Starting with the table renames for the continents:
sales,chicago,*,northamerica,-,- sales,newyork,*,northamerica,-,- sales,london,*,europe,-,- sales,paris,*,europe,-,- sales,munich,*,europe,-,- sales,tokyo,*,asiapac,-,- sales,shanghai,*,asiapac,-,-
A single rule to handle the renaming of any table not explicitly
mentioned in the list above into the
misc schema:
*,*,*,misc,-,-
Now a rule to change the area
column for all tables to region.
This requires a wildcard match against the schema and table names:
*,*,area,-,-,region
And finally the explicit changes for the value column to the corresponding currency:
*,chicago,value,-,-,usd *,newyork,value,-,-,usd *,london,value,-,-,gbp *,paris,value,-,-,eur *,munich,value,-,-,eur *,moscow,value,-,-,rub *,tokyo,value,-,-,jpy *,shanghai,value,-,-,cny *,sydney,value,-,-,aud