6.3. Deploying Multiple Replicators on a Single Host

It is possible to install multiple replicators on the same host. This can be useful, either when building complex topologies with multiple services, and in hetereogenous environments where you are reading from one database and writing to another that may be installed on the same single server.

When installing multiple replicator services on the same host, different values must be set for the following configuration parameters:

6.3.1. Preparing Multiple Replicators

Before continuing with deployment you will need the following:

  1. The name to use for the service.

  2. The list of datasources in the service. These are the servers which will be running MySQL.

  3. The username and password of the MySQL replication user.

All servers must be prepared with the proper prerequisites. See Section 2.3, “Prepare Hosts” and Appendix B, Prerequisites for additional details.

  • RMI network port used for communicating with the replicator service.

    Set through the --rmi-port parameter to tpm. Note that RMI ports are configured in pairs; the default port is 10000, port 10001 is used automatically. When specifying an alternative port, the subsequent port must also be available. For example, specifying port 10002 also requires 10003.

  • THL network port used for exchanging THL data.

    Set through the --thl-port parameter to tpm. The default THL port is 2112. This option is required for services operating as masters (extractors).

  • Master THL port, i.e. the port from which a slave will read THL events from the master

    Set through the --master-thl-port parameter to tpm. When operating as a slave, the explicit THL port should be specified to ensure that you are connecting to the THL port correctly.

  • Master hostname

    Set through the --master-thl-host parameter to tpm. This is optional if the master hostname has been configured correctly through the --master parameter.

  • Installation directory used when the replicator is installed.

    Set through the --install-directory or --install-directory parameters to tpm. This directory must have been created, and be configured with suitable permissions before installation starts. For more information, see Section B.3.3, “Directory Locations and Configuration”.

6.3.2. Install Multiple Replicators

For example, to create two services, one that reads from MySQL and another that writes to MongoDB on the same host:

  1. Install the Tungsten Replicator package or download the Tungsten Replicator tarball, and unpack it:

    shell> cd /opt/continuent/software
    shell> tar zxf tungsten-replicator-6.0.5-40.tar.gz
  2. Create the proper directories with appropriate ownership and permissions:

    shell> sudo mkdir /opt/applier /opt/extractor
    shell> sudo chown tungsten: /opt/applier/ /opt/extractor/
    shell> sudo chmod 700 /opt/applier/ /opt/extractor/
  3. Change to the Tungsten Replicator directory:

    shell> cd tungsten-replicator-6.0.5-40
  4. Extractor reading from MySQL (Click link to switch examples between Staging Method or INI Method):

    Show Staging

    Show INI

    shell> ./tools/tpm configure defaults \
        --reset \
        --install-directory=/opt/extractor \
        --user=tungsten \
        --profile-script=~/.bash_profile \
        --mysql-allow-intensive-checks=true \
        --disable-security-controls=true \
        --executable-prefix=ext
    
    
    shell> ./tools/tpm configure alpha \
        --master=offboardhost \
        --members=offboardhost \
        --enable-heterogeneous-service=true \
        --replication-port=3306 \
        --replication-user=tungsten_alpha \
        --replication-password=secret \
        --datasource-mysql-conf=/etc/my.cnf \
        --svc-extractor-filters=colnames,pkey \
        --property=replicator.filter.pkey.addColumnsToDeletes=true \
        --property=replicator.filter.pkey.addPkeyToInserts=true \
        --mysql-enable-enumtostring=true \
        --mysql-enable-settostring=true \
        --mysql-use-bytes-for-string=false
    
    [defaults]
    install-directory=/opt/extractor
    user=tungsten
    profile-script=~/.bash_profile
    mysql-allow-intensive-checks=true
    disable-security-controls=true
    executable-prefix=ext
    
    
    [alpha]
    master=offboardhost
    members=offboardhost
    enable-heterogeneous-service=true
    replication-port=3306
    replication-user=tungsten_alpha
    replication-password=secret
    datasource-mysql-conf=/etc/my.cnf
    svc-extractor-filters=colnames,pkey
    property=replicator.filter.pkey.addColumnsToDeletes=true
    property=replicator.filter.pkey.addPkeyToInserts=true
    mysql-enable-enumtostring=true
    mysql-enable-settostring=true
    mysql-use-bytes-for-string=false
    

    Configuration group defaults

    The description of each of the options is shown below; click the icon to hide this detail:

    Click the icon to show a detailed description of each argument.

    Configuration group alpha

    The description of each of the options is shown below; click the icon to hide this detail:

    Click the icon to show a detailed description of each argument.

    This is a standard configuration using the default ports, with the directory /opt/extractor.

  5. Applier for writing to MongoDB (Click link to switch examples between Staging Method or INI Method):

    Show Staging

    Show INI

    shell> ./tools/tpm configure defaults \
        --reset \
        --install-directory=/opt/applier \
        --profile-script=~/.bash_profile \
        --skip-validation-check=InstallerMasterSlaveCheck \
        --executable-prefix=app
    
    
    shell> ./tools/tpm configure alpha \
        --master=localhost \
        --members=localhost \
        --role=slave \
        --datasource-type=mongodb \
        --replication-user=tungsten \
        --replication-password=secret \
        --rmi-port=10002 \
        --master-thl-port=2112 \
        --master-thl-host=localhost \
        --thl-port=2113
    
    [defaults]
    install-directory=/opt/applier
    profile-script=~/.bash_profile
    skip-validation-check=InstallerMasterSlaveCheck
    executable-prefix=app
    
    
    [alpha]
    master=localhost
    members=localhost
    role=slave
    datasource-type=mongodb
    replication-user=tungsten
    replication-password=secret
    rmi-port=10002
    master-thl-port=2112
    master-thl-host=localhost
    thl-port=2113
    

    Configuration group defaults

    The description of each of the options is shown below; click the icon to hide this detail:

    Click the icon to show a detailed description of each argument.

    Configuration group alpha

    The description of each of the options is shown below; click the icon to hide this detail:

    Click the icon to show a detailed description of each argument.

    In this configuration, the master THL port is specified explicitly, along with the THL port used by this replicator, the RMI port used for administration, and the installation directory /opt/applier.

  6. Run tpm to install the software

    shell > ./tools/tpm install

    During the startup and installation, tpm will notify you of any problems that need to be fixed before the service can be correctly installed and started. If start-and-report is set and the service starts correctly, you should see the configuration and current status of the service.

  7. Initialize your PATH and environment.

    shell > source /opt/extractor/share/env.sh
    shell > source /opt/applier/share/env.sh

  8. Check the replication status.

When multiple replicators have been installed, checking the replicator status through trepctl depends on the replicator executable location used. If /opt/extractor/tungsten/tungsten-replicator/bin/trepctl, the extractor service status will be reported. If /opt/applier/tungsten/tungsten-replicator/bin/trepctl is used, then the applier service status will be reported.

To make things easier, in the config examples above executable-prefix has been used, which will set up OS aliases. These aliases are setup when you source the relevant env.sh files, this will also happen by default when you login to the host providing profile-script has been specified

The use of the prefix and aliases, then simplifies the use of all executables, for example, based on the setting of executable-prefix in the above config examples, to report the status of the extractor, you can execute:

shell> ext_trepctl status

Or to check the applier service:

shell> app_trepctl status

Alternatively, a specific replicator can be checked by explicitly specifying the RMI port of the service. For example, to check the extractor service:

shell> trepctl -port 10000 status

Or to check the applier service:

shell> trepctl -port 10002 status

When an explicit port has been specified in this way, the executable used is irrelevant. Any valid trepctl instance will work.

Further, either path may be used to get a summary view using multi_trepctl:

shell> /opt/extractor/tungsten/tungsten-replicator/scripts/multi_trepctl
| host   | servicename | role   | state  | appliedlastseqno | appliedlatency |
| host1  | extractor   | master | ONLINE |                0 |          1.724 |
| host1  | applier     | slave  | ONLINE |                0 |          0.000 |

6.3.3. Best Practices: Multiple Replicators

Follow the guidelines in Section 2.2, “Best Practices”.