10.4.42. shardbyseqno.js Filter

Shards within the replicator enable data to be parallelized when they are applied on the Target.

Pre-configured filter name shardbyseqno
JavaScript Filter File tungsten-replicator/support/filters-javascript/shardbyseqno.js
Property prefix replicator.filter.shardbyseqno
Stage compatibility q-to-dbms
tpm Option compatibility --svc-applier-filters
Data compatibility Any event
Parameters
Parameter Type Default Description
shards numeric (none) Number of shards to be used by the applier

The shardbyseqno filter updates the shard ID, which is embedded into the event metadata, by a configurable number of shards, set by the shards parameter in the configuration:

replicator.filter.shardbyseqno=com.continuent.tungsten.replicator.filter.JavaScriptFilter
replicator.filter.shardbyseqno.script=${replicator.home}/samples/extensions/javascript/shardbyseqno.js
replicator.filter.shardbyseqno.shards=10

The filter works by setting the shard ID in the event using the setShardId() method on the event object:

event.setShardId(event.getSeqno() % shards);

Note

Care should be taken with this filter, as it assumes that the events can be applied in a completely random order by blindly updating the shard ID to a computed valued. Sharding in this way is best used when provisioning new Targets.