Skip to main content
Tungsten Replicator

Applier Replicator Service

The applier replicator service reads information from the THL of the source and applies this to a local instance of Hadoop.

important

Installation must take place on a node within the Hadoop cluster. Writing to a remote HDFS filesystem is not currently supported.

  1. Before installing the applier, the following additions need adding to the extractor configuration. Apply the following parameters, update the extractor and then install the applier

    Add the following to /etc/tungsten/tungsten.ini

    [alpha]
    ...
    enable-batch-service=true
    enable-heterogeneous-service=true

    shell> tpm update

  2. The applier can now be configured.

    Unpack the Tungsten Replicator distribution in a staging directory:

    shell> tar zxf tungsten-replicator-8.0.4-132.tar.gz

  3. Change into the staging directory:

    shell> cd tungsten-replicator-8.0.4-132

  4. Create the /etc/tungsten/tungsten.ini using the example below as a template.

    Example tungsten.ini
    [defaults]
    user=tungsten
    install-directory=/opt/continuent
    profile-script=~/.bash_profile
    skip-validation-check=HostsFileCheck
    skip-validation-check=InstallerMasterSlaveCheck
    skip-validation-check=DatasourceDBPort
    skip-validation-check=DirectDatasourceDBPort
    skip-validation-check=ReplicationServicePipelines
    rest-api-admin-user=apiuser
    rest-api-admin-password=secret
    replicator-rest-api-address=0.0.0.0

    [alpha]
    master=host1
    members=host2
    property=replicator.datasource.global.csvType=hive
    property=replicator.stage.q-to-dbms.blockCommitInterval=1s
    property=replicator.stage.q-to-dbms.blockCommitRowCount=1000
    replication-password=secret
    replication-user=tungsten
    batch-enabled=true
    batch-load-language=js 
    batch-load-template=hadoop
    datasource-type=file
    Show argument definitions
    user=tungstenOS System User, for example tungsten. DO NOT use root.
    profile-script=~/.bash_profileAppend commands to include env.sh in this profile script.
    rest-api-admin-user=apiuserSpecify the initial Admin Username for API access.Available from v7.0.0
    rest-api-admin-password=secretSpecify the initial Admin User Password for API access. rest-api-admin-password alias only available from version 7.1.2 onwards.Available from v7.0.0
    replicator-rest-api-address=0.0.0.0Address for the API to bind too.Available from v7.0.0

  5. Once the prerequisites and configuring of the installation has been completed, the software can be installed:

    shell> ./tools/tpm install

If the installation process fails, check the output of the /tmp/tungsten-configure.log file for more information about the root cause.

Once the service has been installed it can be monitored using the trepctl command. See "Management and Monitoring of Hadoop Deployments" for more information. If there are problems during installation, see "Troubleshooting Hadoop Replication".