11.4.41. shardbyrules.js Filter

The shardbyrules filter allows you to specify granular schema and table level rules for sharding of transactions throguh the replicator. This can provide enhanced performance where regular schema only based sharding would not suit the profile of your application.

Pre-configured filter name shardbyrules
JavaScript Filter File tungsten-replicator/support/filters-javascript/shardbyrules.js
Property prefix replicator.filter.shardbyrules
Stage compatibility q-to-dbms
tpm Option compatibility --svc-applier-filters
Data compatibility ROW
Parameters
Parameter Type Default Description
definitionsFile string support/filters-config/shards.json JSON file containing the definition of which events and which tables to skip

Note

For this filter to function, you will need to ensure your database is configured for ROW based binary logging

The key part of the filter is configuring the rules to suit your sharding requirements. Start by copying the sample file and then editing to suit:

shell> cp /opt/continuent/tungsten/tungsten-replicator/support/filters-config/shards.json /opt/continuent/share/shards.json

Within the configuration, you would then specify this definitionsFile:

svc-applier-filters=shardbyrules
property=replicator.filter.shardbyrules.definitionsFile=/opt/continuent/share/shards.json

Example:

{
  "default": "defaultShardName",
  "schemas": [
    {
      "schema": "schemaA",
      "shardId": "MyShard1"
    },
    {
      "schema": "schemaB",
      "shardId": "MyShard2"
    }
  ],
  "tables": [
    {
      "schema": "schemaB",
      "table": "MyTable1",
      "shardId": "MyShard1"
    }
  ]
}
  • schemaA gets assigned to MyShard1

  • schemaB gets assigned to MyShard2

  • With the exception of schemaB.MyTable1, which also gets assigned to MyShard1.

With this behavior, it would be able to exceute concurrently with transactions either hitting other tables from schemaB, or with tables from other schemas.