6.4.4.3. Slave Replicator Service

The slave replicator service reads information from the THL of the master and applies this to a local instance of Hadoop.

Once the service has been installed it can be monitored using the trepctl command. See Section 6.4.4.6, “Management and Monitoring of Hadoop Deployments” for more information. If there are problems during installation, see Section 6.4.4.7, “Troubleshooting Hadoop Replication”.

The slave replicator service reads information from the THL of the master and applies this to a local instance of Hadoop.

Important

Installation must take place on a node within the Hadoop cluster. Writing to a remote HDFS filesystem is not currently supported.

The tpm required to install the slave replicator:

  1. Unpack the Tungsten Replicator distribution in staging directory:

    shell> tar zxf tungsten-replicator-5.3.tar.gz
  2. Change into the staging directory:

    shell> cd tungsten-replicator-5.3
  3. Execute the tpm to perform the installation. The tpm command shown below configures a loading mechanism using files that are copied into HDFS by the replicator.

    shell> ./tools/tpm install alpha \
    --batch-enabled=true \
    --batch-load-language=js \
    --batch-load-template=hadoop \
    --datasource-type=file \
    --install-directory=/opt/continuent \
    --master=host1 \
    --members=host2 \
    --property=replicator.datasource.global.csvType=hive \
    --property=replicator.stage.q-to-dbms.blockCommitInterval=1s \
    --property=replicator.stage.q-to-dbms.blockCommitRowCount=1000 \
    --replication-password=secret \
    --replication-user=tungsten \
    --skip-validation-check=DatasourceDBPort \
    --skip-validation-check=DirectDatasourceDBPort \
    --skip-validation-check=HostsFileCheck \
    --skip-validation-check=InstallerMasterSlaveCheck \
    --skip-validation-check=ReplicationServicePipelines \
    --rmi-port=25550 \
    --start-and-report=true

    The description of each of the options is shown below; click the icon to hide this detail:

    Click the icon to show a detailed description of each argument.

There are optional parameters that can be added to this configuration to set alternative settings and options.

Click the icon to show the detailed list of optional parameters:

For a list of all the optional parameters available, click the icon:

If the installation process fails, check the output of the /tmp/tungsten-configure.log file for more information about the root cause.

Once the service has been installed it can be monitored using the trepctl command. See Section 6.4.4.6, “Management and Monitoring of Hadoop Deployments” for more information. If there are problems during installation, see Section 6.4.4.7, “Troubleshooting Hadoop Replication”.