Shards within the replicator enable data to be parallelized when they are applied on the Target.
Pre-configured filter name |
shardbyseqno
| ||
JavaScript Filter File |
tungsten-replicator/support/filters-javascript/shardbyseqno.js
| ||
Property prefix |
replicator.filter.shardbyseqno
| ||
Stage compatibility |
q-to-dbms
| ||
tpm Option compatibility |
--svc-applier-filters
| ||
Data compatibility | Any event | ||
Parameters | |||
Parameter | Type | Default | Description |
shards
|
numeric
| (none) | Number of shards to be used by the applier |
The shardbyseqno
filter updates the
shard ID, which is embedded into the event metadata, by a configurable
number of shards, set by the
shards
parameter in the
configuration:
replicator.filter.shardbyseqno=com.continuent.tungsten.replicator.filter.JavaScriptFilter replicator.filter.shardbyseqno.script=${replicator.home}/samples/extensions/javascript/shardbyseqno.js replicator.filter.shardbyseqno.shards=10
The filter works by setting the shard ID in the event using the
setShardId()
method on the event object:
event.setShardId(event.getSeqno() % shards);
Care should be taken with this filter, as it assumes that the events can be applied in a completely random order by blindly updating the shard ID to a computed valued. Sharding in this way is best used when provisioning new Targets.