3.3.5. Resetting a single dataservice


The procedures in this section are designed for the Multi-Site/Active-Active topology ONLY. Do NOT use these procedures for Composite Active/Active Clustering uing v6 onwards.

For version 6.x onwards, Composite Active/Active Clustering, please refer to Section 3.4, “Deploying Composite Active/Active Clusters”

Under certain conditions, dataservices in an active/active configuration may drift and/or become inconsistent with the data in another dataservice. If this occurs, you may need to re-provision the data on one or more of the dataservices after first determining the definitive source of the information.

In the following example the west service has been determined to be the definitive copy of the data. To fix the issue, all the datasources in the east service will be reprovisioned from one of the datasources in the west service.

The following is a guide to the steps that should be followed. In the example procedure it is the east service that has failed:

  1. Put the dataservice into MAINTENANCE mode. This ensures that Tungsten Cluster will not attempt to automatically recover the service.

    cctrl [east]> set policy maintenance
  2. On the east, failed, Tungsten Cluster service, put each Tungsten Connector offline:

    cctrl [east]> router * offline
  3. Reset the failed Tungsten Replicator service on all servers connected to the failed Tungsten Cluster service. For example, on west{1,2,3} reset the east Tungsten Replicator service:

    shell west> /opt/replicator/tungsten/tungsten-replicator/bin/trepctl -service east offline
    shell west> /opt/replicator/tungsten/tungsten-replicator/bin/trepctl -service east reset -all -y
  4. Reset the Tungsten Cluster service on each server within the failed region (east{1,2,3}):

    shell east> /opt/continuent/tungsten/tungsten-replicator/bin/replicator stop
    shell east> /opt/continuent/tungsten/tools/tpm reset east
    shell east> /opt/continuent/tungsten/tungsten-replicator/bin/replicator start
  5. Restore a backup on each host (east{1,2,3}) in the failed east service from a host in the west service:

    shell east> /opt/continuent/tungsten/tungsten-replicator/scripts/tungsten_provision_slave \
        --direct --source=west1
  6. Place all the Tungsten Replicator services on west{1,2,3} back online:

    shell west> /opt/replicator/tungsten/tungsten-replicator/bin/trepctl -service east online
  7. On the east, failed, Tungsten Cluster service, put each Tungsten Connector online:

    cctrl [east]> router * online
  8. Set the policy back to automatic:

    cctrl> set policy automatic