Rename Filter

The rename filter enables schemas to be renamed at the database, table and column levels, and for complex combinations of these renaming operations. Configuration is through a CSV file that defines the rename parameters. A single CSV file can contain multiple rename definitions. The rename operations occur only on ROW based events.

The rename filter also performs schema renames on statement data.

Pre-configured filter name	rename
Classname	com.continuent.tungsten.replicator.filter.RenameFilter
Property prefix	replicator.filter.rename
Stage compatibility
tpm Option compatibility
Data compatibility	Row events.

Parameters

Parameter	Type	Default	Description
definitionsFile	string	${replicator.home.dir}/samples/extensions/java/rename.csv	Location of the CSV file that contains the rename definitions.

The CSV file is only read when an explicit reconfigure operation is triggered. If the file is changed, a configure operation (using tpm update) must be initiated to force reconfiguration.

To enable add the following to your /etc/tungsten/tungsten.ini and issue tpm update:

svc-applier-filters=rename

The CSV consists of multiple lines, one line for each rename specification. Comments are supported using the # character.

The format of each line of the CSV is:

originalSchema,originalTable,originalColumn,newSchema,newTable,newColumn

Where:

• originalSchema, originalTable, originalColumn define the original schema, table and column.

  Definition can either be:

  • Explicit schema, table or column name

  • * character, which indicates that all entries should match.

• newSchema, newTable, newColumn define the new schema, table and column for the corresponding original specification.

  Definition can either be:

  • Explicit schema, table or column name

  • - character, which indicates that the corresponding object should not be updated.

For example, the specification:

*,chicago,*,-,newyork,-

Would rename the table chicago in every database schema to newyork. The schema and column names are not modified.

The specification:

*,chicago,destination,-,-,source

Would match all schemas, but update the column destination in the table chicago to the column name source, without changing the schema or table name.

Processing of the individual rules is executed in a specific order to allow for complex matching and application of the rename changes.

• Rules are case sensitive.

• Schema names are looked up in the following order:

  1. schema.table (explicit schema/table)

  2. schema.* (explicit schema, wildcard table)

• Table names are looked up in the following order:

  1. schema.table (explicit schema/table)

  2. *.table (wildcard schema, explicit table)

• Column names are looked up in the following order:

  1. schema.table (explicit schema/table)

  2. schema.* (explicit schema, wildcard table)

  3. *.table (wildcard schema, explicit table)

  4. . (wildcard schema, wildcard table)

• Rename operations match the first specification according to the above rules, and only one matching rule is executed.

Rename Filter Examples

When processing multiple entries that would match the same definition, the above ordering rules are applied. For example, the definition:

asia,*,*,america,-,-
asia,shanghai,*,europe,-,-

Would rename asia.shanghai to europe.shanghai, while renaming all other tables in the schema asia to the schema america. This is because the explicit schema.table rule is matched first and then executed.

Complex renames involving multiple schemas, tables and columns can be achieved by writing multiple rules into the same CSV file. For example given a schema where all the tables currently reside in a single schema, but must be renamed to specific continents, or to a 'miscellaneous' schema, while also updating the column names to be more neutral would require a detailed rename definition.

Existing tables are in the schema sales:

chicago
newyork
london
paris
munich
moscow
tokyo
shanghai
sydney

Need to be renamed to:

northamerica.chicago
northamerica.newyork
europe.london
europe.paris
europe.munich
misc.moscow
asiapac.tokyo
asiapac.shanghai
misc.sydney

Meanwhile, the table definition needs to be updated to support more complex structure:

id
area
country
city
value
type

The area is being updated to contain the region within the country, while the value should be renamed to the three-letter currency code, for example, the london table would rename the value column to gbp.

The definition can be divided up into simple definitions at each object level, relying on the processing order to handle the individual exceptions. Starting with the table renames for the continents:

sales,chicago,*,northamerica,-,-
sales,newyork,*,northamerica,-,-
sales,london,*,europe,-,-
sales,paris,*,europe,-,-
sales,munich,*,europe,-,-
sales,tokyo,*,asiapac,-,-
sales,shanghai,*,asiapac,-,-

A single rule to handle the renaming of any table not explicitly mentioned in the list above into the misc schema:

*,*,*,misc,-,-

Now a rule to change the area column for all tables to region. This requires a wildcard match against the schema and table names:

*,*,area,-,-,region

And finally the explicit changes for the value column to the corresponding currency:

*,chicago,value,-,-,usd
*,newyork,value,-,-,usd
*,london,value,-,-,gbp
*,paris,value,-,-,eur
*,munich,value,-,-,eur
*,moscow,value,-,-,rub
*,tokyo,value,-,-,jpy
*,shanghai,value,-,-,cny
*,sydney,value,-,-,aud