Continuent Documentation

Search:

11.4.35. Replicate Filter

The rowadddbname filter adds a new column to every incoming row of data containing the schema name of the table. This can used in combination with analytics replication targets where the information is being written into a single schema, concentrating data from multiple source databases into a single database. Optionally, the filter can also add the information as a primary key (required for some heterogeneous targets), and the hash of the source database name.

Pre-configured filter name rowadddbname
Classname com.continuent.tungsten.replicator.filter.JavaScriptFilter
Property prefix replicator.filter.rowadddbname
Stage compatibility Any
tpm Option compatibility  
Data compatibility Row-events only
Parameters
Parameter Type Default Description
adddbhash boolean true Add a hash column, computed using the Java hash value of the string for the incoming source database name
addkey boolean true Add the information to the primary key data in the THL, in addition to the column data
fieldname string dbname The name of the database name column that will be added
fieldhashname string dbname The name of the database has name column that will be added

The rowadddbname filter adds a column to every row of THL processed by the filter. This can be used when data is being written into a single target schema within an analytics environment, and where the source database can be used to identify a customer, project or dataset, and therefore queried within the analytics platform either specifically or with other datasets.

The filter is able to perform the following modifications to every row of incoming data:

  • Add the source database or schema name.

  • Add a numerical hash value of the string of the source database of schema name.

  • Add the database name (and hash name) to the primary key data.

For example, the source THL:

- ROW# = 0
  - COL(1: id) = 12
  - COL(2: msg) = Hello
  - COL(3: msg2) = World
  - KEY(1: id) = NULL

And after the filter has been applied:

- ROW# = 0
  - COL(1: id) = 12
  - COL(2: msg) = Hello
  - COL(3: msg2) = World
  - COL(4: dbname) = msg
  - COL(5: dbname_hash) = 108417
  - KEY(1: id) = NULL
  - KEY(4: dbname) = NULL
  - KEY(5: dbname_hash) = NULL

This filter is a required component of deployments when replicating into a single schema within Vertica (see Section 6.8.3, “Concentrating Data into a Single Schema with Vertica”) and Amazon Redshift (see Section 6.8.3, “Concentrating Data into a Single Schema with Vertica”).