Skip to main content
Tungsten Replicator

Replicating Data Out of a Cluster

If you have an existing cluster and you want to replicate the data out to a separate standalone server using Tungsten Replicator then you can create a cluster alias, and use a primary/replica topology to replicate from the cluster. This allows for THL events from the cluster to be applied to a separate server for the purposes of backup or separate analysis.

Topologies: Replicating Data Out of a Cluster

During the installation process topology=cluster-alias and topology=cluster-slave are declared. The cluster-alias describes all of the servers in the cluster and how they may be reached. The cluster-slave defines one or more servers that will replicate from the cluster.

The Tungsten Replicator will be installed on the cluster-slave server. That server will download THL data and apply them to the local server. If the cluster-slave has more than one server; one of them will be declared the relay (or primary). The other members of the cluster-slave may also download THL data from that server.

If the relay for the cluster-slave fails; the other nodes will automatically start downloading THL data from a server in the cluster. If a non-relay server fails; it will not have any impact on the other members.

Prepare​

  1. Identify the cluster to replicate from. You will need the primary, replicas and THL port (if specified). Use tpm reverse from a cluster member to find the correct values.

  2. If you are replicating to a non-MySQL server. Update the configuration of the cluster to include the following properties prior to beginning.

    svc-extractor-filters=colnames,pkey
    property=replicator.filter.pkey.addColumnsToDeletes=true
    property=replicator.filter.pkey.addPkeyToInserts=true

  3. Identify all servers that will replicate from the cluster. If there is more than one, a relay server should be identified to replicate from the cluster and provide THL data to other servers.

  4. Prepare each server according to the prerequisites for the DBMS platform it is serving. If you are working with multiple DBMS platforms; treat each platform as a different "cluster-slave" during deployment.

  5. Make sure the THL port for the cluster is open between all servers.

Deploy​

  1. Install the Tungsten Replicator RPM package or download the Tungsten Replicator tarball, and unpack it:

    shell> cd /opt/continuent/software
    shell> tar zxf tungsten-replicator-8.0.4-132.tar.gz

  2. Change to the unpackaged directory:

    shell> cd tungsten-replicator-8.0.4-132

  3. Configure the replicator

    Example tungsten.ini
    [defaults]
    install-directory=/opt/continuent
    profile-script=~/.bash_profile
    replication-password=secret
    replication-port=13306
    replication-user=tungsten
    user=tungsten
    rest-api-admin-user=apiuser
    rest-api-admin-password=secret
    replicator-rest-api-address=0.0.0.0

    [alpha]
    master=host1
    members=host1,host2,host3
    thl-port=2112
    topology=cluster-alias

    [beta]
    relay=host6
    relay-source=alpha
    topology=cluster-slave
    Show argument definitions
    profile-script=~/.bash_profileAppend commands to include env.sh in this profile script.
    replication-user=tungstenUser for database connection.
    user=tungstenOS System User, for example tungsten. DO NOT use root.
    rest-api-admin-user=apiuserSpecify the initial Admin Username for API access.Available from v7.0.0
    rest-api-admin-password=secretSpecify the initial Admin User Password for API access. rest-api-admin-password alias only available from version 7.1.2 onwards.Available from v7.0.0
    replicator-rest-api-address=0.0.0.0Address for the API to bind too.Available from v7.0.0
    Important

    If you are replicating to a non-MySQL server. Include the following steps in your configuration.

    shell> mkdir -p /opt/continuent/share/
    shell> cp tungsten-replicator/support/filters-config/convertstringfrommysql.json /opt/continuent/share/

    Then, include the following parameters in the configuration

    property=replicator.stage.remote-to-thl.filters=convertstringfrommysql
    property=replicator.filter.convertstringfrommysql.definitionsFile=/opt/continuent/share/convertstringfrommysql.json
    Important

    The dataservice cluster-alias name MUST be the same as the cluster dataservice name that you are replicating from.

    Note

    Do not include start-and-report if you are taking over for MySQL native replication. See "operations-seeddata-mysqlnative" for next steps after completing installation.

  4. Once the configuration has been completed, you can perform the installation to set up the services using this configuration:

    shell> ./tools/tpm install

During the installation and startup, tpm will notify you of any problems that need to be fixed before the service can be correctly installed and started. If the service starts correctly, you should see the configuration and current status of the service.

If the installation process fails, check the output of the /tmp/tungsten-configure.log file for more information about the root cause.

The replicator should be installed and ready to use.