Continuent Documentation

Search:

10.4.28. OptimizeUpdates Filter

The optimizeupdates filter works with row-based events to simplify the update statement and remove columns/values that have not changed. This reduces the workload and row data exchanged between replicators.

Pre-configured filter name optimizeupdates
Classname com.continuent.tungsten.replicator.filter.OptimizeUpdatesFilter
Property prefix replicator.filter.optimizeupdates
Stage compatibility  
tpm Option compatibility  
Data compatibility Row events
Parameters
(none)

The filter operates by removing column values for keys in the update statement that do not change. For example, when replicating the row event from the statement:

mysql>  update testopt set msg = 'String1', string = 'String3' where id = 1;

Generates the following THL event data:

- SQL(0) =
 - ACTION = UPDATE
 - SCHEMA = test
 - TABLE = testopt
 - ROW# = 0
  - COL(1: id) = 1
  - COL(2: msg) = String1
  - COL(3: string) = String3
  - KEY(1: id) = 1

Column 1 (id) in this case is automatically implied by the KEY entry required for the update.

With the optimizeupdates filter enabled, the data in the THL is simplified to:

- SQL(0) =
 - ACTION = UPDATE
 - SCHEMA = test
 - TABLE = testopt
 - ROW# = 0
  - COL(2: msg) = String1
  - COL(3: string) = String4
  - KEY(1: id) = 1

In tables where there are multiple keys the stored THL information can be reduced further.

Warning

The filter works by comparing the value of each KEY and COL entry in the THL and determining whether the value has changed or not. If the number of keys and columns do not match then the filter will fail with the following error message:

Caused by: java.lang.Exception: Column and key count is different in this event! Cannot filter

This may be due to a filter earlier within the filter configuration that has optimized or simplified the data. For example, the pkey filter removes KEY entries from the THL that are not primary keys, or dropcolumn which drops column data.