4.4. Deploying Multiple Replicators on a Single Host

It is possible to install multiple replicators on the same host. This can be useful, either when building complex topologies with multiple services, and in hetereogenous environments where you are reading from one database and writing to another that may be installed on the same single server.

When installing multiple replicator services on the same host, different values must be set for the following configuration parameters:

4.4.1. Prepare: Multiple Replicators

Before continuing with deployment you will need the following:

  1. The name to use for the service.

  2. The list of datasources in the service. These are the servers which will be running MySQL.

  3. The username and password of the MySQL replication user.

All servers must be prepared with the proper prerequisites. See Section 2.3, “Prepare Hosts” and Appendix C, Prerequisites for additional details.

  • RMI network port used for communicating with the replicator service.

    Set through the --rmi-port parameter to tpm. Note that RMI ports are configured in pairs; the default port is 10000, port 10001 is used automatically. When specifying an alternative port, the subsequent port must also be available. For example, specifying port 10002 also requires 10003.

  • THL network port used for exchanging THL data.

    Set through the --thl-port parameter to tpm. The default THL port is 2112. This option is required for services operating as masters (extractors).

  • Master THL port, i.e. the port from which a slave will read THL events from the master

    Set through the --master-thl-port parameter to tpm. When operating as a slave, the explicit THL port should be specified to ensure that you are connecting to the THL port correctly.

  • Master hostname

    Set through the --master-thl-host parameter to tpm. This is optional if the master hostname has been configured correctly through the --master parameter.

  • Installation directory used when the replicator is installed.

    Set through the --install-directory or --install-directory parameters to tpm. This directory must have been created, and be configured with suitable permissions before installation starts. For more information, see Section C.3.3, “Directory Locations and Configuration”.

4.4.2. Install: Multiple Replicators

4.4.2.1. Deploying Multiple Replicators on a Single Host (Staging Use Case)

For example, to create two services, one that reads from MySQL and another that writes to MongoDB on the same host:

  1. Install the Tungsten Replicator package or download the Tungsten Replicator tarball, and unpack it:

    shell> cd /opt/continuent/software
    shell> tar zxf tungsten-replicator-5.0.0-0.tar.gz
  2. Change to the Tungsten Replicator directory:

    shell> cd tungsten-replicator-5.0.0-0
  3. Extractor reading from MySQL:

    shell> ./tools/tpm configure mysql2mongodb \
    --install-directory=/opt/extractor \
    --java-file-encoding=UTF8 \
    --master=host1 \
    --members=host1 \
    --mysql-enable-enumtostring=true \
    --mysql-enable-settostring=true \
    --mysql-use-bytes-for-string=false \
    --replication-password=password \
    --replication-user=tungsten \
    --start=true \
    --svc-extractor-filters=colnames,pkey

    This is a standard configuration using the default ports, with the directory /opt/extractor.

  4. Reset the configuration:

    shell> ./tools/tpm configure defaults --reset
  5. Applier for writing to MongoDB:

    shell> ./tools/tpm configure mysql2mongodb \
    --datasource-type=mongodb \
    --role=slave \
    --install-directory=/opt/applier \
    --java-file-encoding=UTF8 \
    --master=host1 \
    --members=host1 \
    --skip-validation-check=InstallerMasterSlaveCheck \
    --start=true \
    --svc-parallelization-type=none \
    --topology=master-slave \
    --rmi-port=10002 \
    --master-thl-port=2112 \
    --master-thl-host=host1 \
    --thl-port=2113

    In this configuration, the master THL port is specified explicitly, along with the THL port used by this replicator, the RMI port used for administration, and the installation directory /opt/applier.

When multiple replicators have been installed, checking the replicator status through trepctl depends on the replicator executable location used. If /opt/extractor/tungsten/tungsten-replicator/bin/trepctl, the extractor service status will be reported. If /opt/applier/tungsten/tungsten-replicator/bin/trepctl is used, then the applier service status will be reported.

Alternatively, a specific replicator can be checked by explicitly specifying the RMI port of the service. For example, to check the extractor service:

shell> trepctl -port 10000 status

Or to check the applier service:

shell> trepctl -port 10002 status

When an explicit port has been specified in this way, the executable used is irrelevant. Any valid trepctl instance will work.

Further, either path may be used to get a summary view using multi_trepctl:

shell> /opt/extractor/tungsten/tungsten-replicator/scripts/multi_trepctl
| host   | servicename | role   | state  | appliedlastseqno | appliedlatency |
| host1  | extractor   | master | ONLINE |                0 |          1.724 |
| host1  | applier     | slave  | ONLINE |                0 |          0.000 |

4.4.2.2. Deploying Multiple Replicators on a Single Host (INI Use Case)

It is possible to install multiple replicators on the same host. This can be useful, either when building complex topologies with multiple services, and in hetereogenous environments where you are reading from one database and writing to another that may be installed on the same single server.

When installing multiple replicator services on the same host, different values must be set for the following configuration parameters:

  • RMI network port used for communicating with the replicator service.

    Set through the rmi-port parameter to tpm. Note that RMI ports are configured in pairs; the default port is 10000, port 10001 is used automatically. When specifying an alternative port, the subsequent port must also be available. For example, specifying port 10002 also requires 10003.

  • THL network port used for exchanging THL data.

    Set through the thl-port parameter to tpm. The default THL port is 2112. This option is required for services operating as masters (extractors).

  • Master THL port, i.e. the port from which a slave will read THL events from the master

    Set through the master-thl-port parameter to tpm. When operating as a slave, the explicit THL port should be specified to ensure that you are connecting to the THL port correctly.

  • Master hostname

    Set through the master-thl-host parameter to tpm. This is optional if the master hostname has been configured correctly through the master parameter.

  • Installation directory used when the replicator is installed.

    Set through the install-directory or install-directory parameters to tpm. This directory must have been created, and be configured with suitable permissions before installation starts. For more information, see Section C.3.3, “Directory Locations and Configuration”.

For example, to create two services, one that reads from MySQL and another that writes to MongoDB on the same host:

  1. Install the Tungsten Replicator™ package (.rpm), or download the compressed tarball and unpack it:

    shell> cd /opt/continuent/software
    shell> tar zxf tungsten-replicator-5.0.0-0.tar.gz
  2. Change to the Tungsten Replicator directory:

    shell> cd tungsten-replicator-5.0.0-0
  3. Create the proper directories with appropriate ownership and permissions:

    shell> sudo mkdir /opt/applier /opt/extractor
    shell> sudo chown tungsten: /opt/applier/ /opt/extractor/
    shell> sudo chmod 700 /opt/applier/ /opt/extractor/
  4. Create /etc/tungsten/tungsten-extractor.ini with the following configuration:

    [mysql2mongodb]
    install-directory=/opt/extractor
    master=host1
    members=host1
    mysql-enable-enumtostring=true
    mysql-enable-settostring=true
    mysql-use-bytes-for-string=false
    replication-password=password
    replication-user=tungsten
    svc-extractor-filters=colnames,pkey
    java-file-encoding=UTF8
    start-and-report=true
    

    The description of each of the options is shown below; click the icon to hide this detail:

    Click the icon to show a detailed description of each argument.

  5. Create /etc/tungsten/tungsten-applier.ini with the following configuration:

    [mysql2mongodb]
    install-directory=/opt/applier
    topology=master-slave
    role=slave
    datasource-type=mongodb
    master=host1
    members=host1
    skip-validation-check=InstallerMasterSlaveCheck
    svc-parallelization-type=none
    master-thl-host=host1
    master-thl-port=2112
    thl-port=2113
    rmi-port=10002
    java-file-encoding=UTF8
    start-and-report=true
    

    The description of each of the options is shown below; click the icon to hide this detail:

    Click the icon to show a detailed description of each argument.

  6. Run tpm to install the software with the INI-based configuration:

    shell > ./tools/tpm install

    During the startup and installation, tpm will notify you of any problems that need to be fixed before the service can be correctly installed and started. If start-and-report is set and the service starts correctly, you should see the configuration and current status of the service.

  7. Initialize your PATH and environment.

    shell > source /opt/continuent/share/env.sh

  8. Check the replication status.

    When multiple replicators have been installed, checking the replicator status through trepctl depends on the replicator executable location used. If /opt/extractor/tungsten/tungsten-replicator/bin/trepctl, the extractor service status will be reported. If /opt/applier/tungsten/tungsten-replicator/bin/trepctl is used, then the applier service status will be reported.

    Alternatively, a specific replicator can be checked by explicitly specifying the RMI port of the service. For example, to check the extractor service:

    shell> trepctl -port 10000 status

    Or to check the applier service:

    shell> trepctl -port 10002 status

    When an explicit port has been specified in this way, the executable used is irrelevant. Any valid trepctl instance will work.

    Further, either path may be used to get a summary view using multi_trepctl:

    shell> /opt/extractor/tungsten/tungsten-replicator/scripts/multi_trepctl
    | host   | servicename | role   | state  | appliedlastseqno | appliedlatency |
    | host1  | extractor   | master | ONLINE |                0 |          1.724 |
    | host1  | applier     | slave  | ONLINE |                0 |          0.000 |

4.4.3. Best Practices: Multiple Replicators

Follow the guidelines in Section 2.2, “Best Practices”.