Chapter 10. Replication Filters
Filtering operates by applying the filter within one, or more, of the stages
configured within the replicator. Stages are the individual steps that occur
within a pipeline, that take information from a source (such as MySQL binary
log) and write that information to an internal queue, the transaction
history log, or apply it to a database. Where the filters are applied
ultimately affect how the information is stores, used, or represented to the
next stage or pipeline in the system.
For example, a filter that removed out all the tables from a specific
database would have different effects depending on the stage it was applied.
If the filter was applied on the master before writing the information into
the THL, then no slave could ever access the table data, because the
information would never be stored into the THL to be transferred to the
slaves. However, if the filter was applied on the slave, then some slaves
could replicate the table and database information, while other slaves could
choose to ignore them. The filtering process also has an impact on other
elements of the system. For example, filtering on the master may reduce
network overhead, albeit at a reduction in the flexibility of the data
In a standard replicator configuration with MySQL, the following stages are
configured in the master, as shown in
Figure 10.1, “Filters: Pipeline Stages on Masters”.
Figure 10.1. Filters: Pipeline Stages on Masters
Within the slave, the stages configured by default are shown in
Figure 10.2, “Filters: Pipeline Stages on Slaves”.
Figure 10.2. Filters: Pipeline Stages on Slaves
Remote THL information is read from a master datasource and written to a
local file on disk.
The THL information is read from the file on disk and stored in an
The data from the in-memory queue is written to the target database.
Filters can be applied during any configured stage, and where the filter is
applied alters the content and availability of the information. The staging
and filtering mechanism can also be used to apply multiple filters to the
data, altering content when it is read and when it is applied.
Where more than one filter is configured for a pipeline, each filter is
executed in the order it appears in the configuration. For example, within
the following fragment:
settostring is executed first,
For certain filter combinations this order can be significant. Some filters
rely on the information provided by earlier filters.