The optimizeupdates filter works
with row-based events to simplify the update statement and remove
columns/values that have not changed. This reduces the workload and row
data exchanged between replicators.
| Pre-configured filter name |
optimizeupdates
| ||
| Classname |
com.continuent.tungsten.replicator.filter.OptimizeUpdatesFilter
| ||
| Property prefix |
replicator.filter.optimizeupdates
| ||
| Stage compatibility | |||
| tpm Option compatibility | |||
| Data compatibility | Row events | ||
| Parameters | |||
| (none) | |||
The filter operates by removing column values for keys in the update statement that do not change. For example, when replicating the row event from the statement:
mysql> update testopt set msg = 'String1', string = 'String3' where id = 1;Generates the following THL event data:
- SQL(0) = - ACTION = UPDATE - SCHEMA = test - TABLE = testopt - ROW# = 0 - COL(1: id) = 1 - COL(2: msg) = String1 - COL(3: string) = String3 - KEY(1: id) = 1
Column 1 (id) in this case is
automatically implied by the KEY entry required for the update.
With the optimizeupdates filter
enabled, the data in the THL is simplified to:
- SQL(0) = - ACTION = UPDATE - SCHEMA = test - TABLE = testopt - ROW# = 0 - COL(2: msg) = String1 - COL(3: string) = String4 - KEY(1: id) = 1
In tables where there are multiple keys the stored THL information can be reduced further.
The filter works by comparing the value of each KEY and COL entry in the THL and determining whether the value has changed or not. If the number of keys and columns do not match then the filter will fail with the following error message:
Caused by: java.lang.Exception: Column and key count is different in this event! Cannot filter
This may be due to a filter earlier within the filter configuration that
has optimized or simplified the data. For example, the
pkey filter removes KEY entries
from the THL that are not primary keys, or
dropcolumn which drops column
data.
The following error message may appear in the logs and in the output from trepctl status to indicate that this ordering issue may be the problem:
OptimizeUpdatesFilter cannot filter, because column and key count is different. Make sure that it is defined before filters which remove keys (eg. PrimaryKeyFilter).