It is possible to install multiple replicators on the same host. This can be useful, either when building complex topologies with multiple services, and in hetereogenous environments where you are reading from one database and writing to another that may be installed on the same single server.
When installing multiple replicator services on the same host, different values must be set for the following configuration parameters:
Before continuing with deployment you will need the following:
The name to use for the service.
The list of datasources in the service. These are the servers which will be running MySQL.
The username and password of the MySQL replication user.
All servers must be prepared with the proper prerequisites. See Appendix B, Prerequisites for additional details.
RMI network port used for communicating with the replicator service.
Set through the --rmi-port parameter
to tpm. Note that RMI ports are configured in
pairs; the default port is 10000, port 10001 is used automatically.
When specifying an alternative port, the subsequent port must also be
available. For example, specifying port 10002 also requires 10003.
THL network port used for exchanging THL data.
Set through the --thl-port parameter
to tpm. The default THL port is 2112. This option
is required for services operating as Extractors.
Extractor THL port, i.e. the port from which an Applier will read THL events from the Extractor
Set through the --master-thl-port
parameter to tpm. When operating as an Applier, the
explicit THL port should be specified to ensure that you are
connecting to the THL port correctly.
Extractor hostname
Set through the --master-thl-host
parameter to tpm. This is optional if the Extractor
hostname has been configured correctly through the
--master parameter.
Installation directory used when the replicator is installed.
Set through the --install-directory
or --install-directory parameters to
tpm. This directory must have been created, and be
configured with suitable permissions before installation starts. For
more information, see Section B.3.4, “Directory Locations and Configuration”.
For example, to create two services, one that reads from MySQL and another that writes to MongoDB on the same host:
Install the Tungsten Replicator package or download the Tungsten Replicator tarball, and unpack it:
shell>cd /opt/continuent/softwareshell>tar zxftungsten-replicator-6.1.25-6.tar.gz
Create the proper directories with appropriate ownership and permissions:
shell>sudo mkdir /opt/applier /opt/extractorshell>sudo chown tungsten: /opt/applier/ /opt/extractor/shell>sudo chmod 700 /opt/applier/ /opt/extractor/
Change to the Tungsten Replicator directory:
shell> cd tungsten-replicator-6.1.25-6Extractor reading from MySQL (Click link to switch examples between Staging Method or INI Method):
shell>./tools/tpm configure defaults \ --reset \ --install-directory=/opt/extractor \ --user=tungsten \ --profile-script=~/.bash_profile \ --mysql-allow-intensive-checks=true \ --disable-security-controls=true \ --executable-prefix=ext \ --rest-api-admin-user=apiuser \ --rest-api-admin-pass=secretshell>./tools/tpm configure alpha \ --master=offboardhost \ --members=offboardhost \ --enable-heterogeneous-service=true \ --replication-port=3306 \ --replication-user=tungsten_alpha \ --replication-password=secret \ --datasource-mysql-conf=/etc/my.cnf \ --svc-extractor-filters=colnames,pkey \ --property=replicator.filter.pkey.addColumnsToDeletes=true \ --property=replicator.filter.pkey.addPkeyToInserts=true \ --mysql-enable-enumtostring=true \ --mysql-enable-settostring=true \ --mysql-use-bytes-for-string=false
shell> vi /etc/tungsten/tungsten.ini[defaults] install-directory=/opt/extractor user=tungsten profile-script=~/.bash_profile mysql-allow-intensive-checks=true disable-security-controls=true executable-prefix=ext rest-api-admin-user=apiuser rest-api-admin-pass=secret[alpha] master=offboardhost members=offboardhost enable-heterogeneous-service=true replication-port=3306 replication-user=tungsten_alpha replication-password=secret datasource-mysql-conf=/etc/my.cnf svc-extractor-filters=colnames,pkey property=replicator.filter.pkey.addColumnsToDeletes=true property=replicator.filter.pkey.addPkeyToInserts=true mysql-enable-enumtostring=true mysql-enable-settostring=true mysql-use-bytes-for-string=false
Configuration group defaults
The description of each of the options is shown below; click the icon to hide this detail:
For staging configurations, deletes all pre-existing configuration information between updating with the new configuration values.
--install-directory=/opt/extractor
install-directory=/opt/extractor
Path to the directory where the active deployment will be installed. The configured directory will contain the software, THL and relay log information unless configured otherwise.
System User
--profile-script=~/.bash_profile
profile-script=~/.bash_profile
Append commands to include env.sh in this profile script
--mysql-allow-intensive-checks=true
mysql-allow-intensive-checks=true
For MySQL installation, enables detailed checks on the supported data types within the MySQL database to confirm compatibility. This includes checking each table definition individually for any unsupported data types.
--disable-security-controls=true
disable-security-controls=true
Disables all forms of security, including SSL, TLS and authentication
When enabled, the supplied prefix is added to each command alias
that is generated for a given installation. This enables
multiple installations to co-exist and and be accessible through
a unique alias. For example, if the executable prefix is
configured as east, then
an alias for the installation to trepctl will
be created as east_trepctl.
Alias information for executable prefix data is stored within
the
$CONTINUENT_ROOT/share/aliases.sh
file for each installation.
--rest-api-admin-user=apiuser
rest-api-admin-user=apiuser
--rest-api-admin-pass=secret
rest-api-admin-pass=secret
Configuration group alpha
The description of each of the options is shown below; click the icon to hide this detail:
The hostname of the primary (extractor) within the current service.
Hostnames for the dataservice members
--enable-heterogeneous-service=true
enable-heterogeneous-service=true
On a Primary
--mysql-use-bytes-for-string
is set to false.
colnames filter is
enabled (in the
binlog-to-q stage
to add column names to the THL information.
pkey filter is
enabled (in the
binlog-to-q and
q-to-dbms stage),
with the
addPkeyToInserts
and
addColumnsToDeletes
filter options set to false.
enumtostring
filter is enabled (in the
q-to-thl stage), to
translate ENUM
values to their string equivalents.
settostring filter
is enabled (in the
q-to-thl stage), to
translate SET
values to their string equivalents.
On a Replica
--mysql-use-bytes-for-string
is set to true.
The network port used to connect to the database server. The default port used depends on the database being configured.
--replication-user=tungsten_alpha
replication-user=tungsten_alpha
For databases that required authentication, the username to use when connecting to the database using the corresponding connection method (native, JDBC, etc.).
The password to be used when connecting to the database using
the corresponding
--replication-user.
--datasource-mysql-conf=/etc/my.cnf
datasource-mysql-conf=/etc/my.cnf
MySQL config file
--svc-extractor-filters=colnames,pkey
svc-extractor-filters=colnames,pkey
Replication service extractor filters
--mysql-enable-enumtostring=true
mysql-enable-enumtostring=true
Enable a filter to convert ENUM values to strings
--mysql-enable-settostring=true
Enable a filter to convert SET types to strings
--mysql-use-bytes-for-string=false
mysql-use-bytes-for-string=false
Transfer strings as their byte representation?
This is a standard configuration using the default ports, with the
directory /opt/extractor.
Applier for writing to MongoDB (Click link to switch examples between Staging Method or INI Method):
shell>./tools/tpm configure defaults \ --reset \ --install-directory=/opt/applier \ --profile-script=~/.bash_profile \ --skip-validation-check=InstallerMasterSlaveCheck \ --executable-prefix=app \ --rest-api-admin-user=apiuser \ --rest-api-admin-pass=secretshell>./tools/tpm configure alpha \ --master=localhost \ --members=localhost \ --role=slave \ --datasource-type=mongodb \ --replication-user=tungsten \ --replication-password=secret \ --rmi-port=10002 \ --master-thl-port=2112 \ --master-thl-host=localhost \ --thl-port=2113
shell> vi /etc/tungsten/tungsten.ini[defaults] install-directory=/opt/applier profile-script=~/.bash_profile skip-validation-check=InstallerMasterSlaveCheck executable-prefix=app rest-api-admin-user=apiuser rest-api-admin-pass=secret[alpha] master=localhost members=localhost role=slave datasource-type=mongodb replication-user=tungsten replication-password=secret rmi-port=10002 master-thl-port=2112 master-thl-host=localhost thl-port=2113
Configuration group defaults
The description of each of the options is shown below; click the icon to hide this detail:
For staging configurations, deletes all pre-existing configuration information between updating with the new configuration values.
--install-directory=/opt/applier
install-directory=/opt/applier
Path to the directory where the active deployment will be installed. The configured directory will contain the software, THL and relay log information unless configured otherwise.
--profile-script=~/.bash_profile
profile-script=~/.bash_profile
Append commands to include env.sh in this profile script
--skip-validation-check=InstallerMasterSlaveCheck
skip-validation-check=InstallerMasterSlaveCheck
The --skip-validation-check
disables a given validation check. If any validation check
fails, the installation, validation or configuration will
automatically stop.
Using this option enables you to bypass the specified check, although skipping a check may lead to an invalid or non-working configuration.
You can identify a given check if an error or warning has been raised during configuration. For example, the default table type check:
... ERROR >> centos >> The datasource root@centos:3306 (WITH PASSWORD) » uses MyISAM as the default storage engine (MySQLDefaultTableTypeCheck) ...
The check in this case is
MySQLDefaultTableTypeCheck,
and could be ignored using
--skip-validation-check=MySQLDefaultTableTypeCheck.
Setting both
--skip-validation-check and
--enable-validation-check is
equivalent to explicitly disabling the specified check.
When enabled, the supplied prefix is added to each command alias
that is generated for a given installation. This enables
multiple installations to co-exist and and be accessible through
a unique alias. For example, if the executable prefix is
configured as east, then
an alias for the installation to trepctl will
be created as east_trepctl.
Alias information for executable prefix data is stored within
the
$CONTINUENT_ROOT/share/aliases.sh
file for each installation.
--rest-api-admin-user=apiuser
rest-api-admin-user=apiuser
--rest-api-admin-pass=secret
rest-api-admin-pass=secret
Configuration group alpha
The description of each of the options is shown below; click the icon to hide this detail:
The hostname of the primary (extractor) within the current service.
Hostnames for the dataservice members
What is the replication role for this service?
Database type
For databases that required authentication, the username to use when connecting to the database using the corresponding connection method (native, JDBC, etc.).
The password to be used when connecting to the database using
the corresponding
--replication-user.
Replication RMI listen port
Primary THL Port
Primary THL Hostname
Port to use for THL Operations
In this configuration, the Extractor THL port is specified explicitly,
along with the THL port used by this replicator, the RMI port used for
administration, and the installation directory
/opt/applier.
Run tpm to install the software
shell > ./tools/tpm install
During the startup and installation, tpm will
notify you of any problems that need to be fixed before the service
can be correctly installed and started. If
start-and-report is set and the
service starts correctly, you should see the configuration and current
status of the service.
Initialize your PATH and environment.
shell >source /opt/extractor/share/env.shshell >source /opt/applier/share/env.sh
Check the replication status.
When multiple replicators have been installed, checking the replicator
status through trepctl depends on the replicator
executable location used. If
/opt/extractor/tungsten/tungsten-replicator/bin/trepctl,
the extractor service status will be reported. If
/opt/applier/tungsten/tungsten-replicator/bin/trepctl
is used, then the applier service status will be reported.
To make things easier, in the config examples above
executable-prefix has been used, which will
set up OS aliases. These aliases are setup when you source the relevant
env.sh files, this will also happen by default when you
login to the host providing profile-script
has been specified
The use of the prefix and aliases, then simplifies the use of all
executables, for example, based on the setting of
executable-prefix in the above config
examples, to report the status of the extractor, you
can execute:
shell> ext_trepctl statusOr to check the applier service:
shell> app_trepctl statusAlternatively, a specific replicator can be checked by explicitly specifying the RMI port of the service. For example, to check the extractor service:
shell> trepctl -port 10000 statusOr to check the applier service:
shell> trepctl -port 10002 statusWhen an explicit port has been specified in this way, the executable used is irrelevant. Any valid trepctl instance will work.
Further, either path may be used to get a summary view using multi_trepctl:
shell> /opt/extractor/tungsten/tungsten-replicator/scripts/multi_trepctl
| host | servicename | role | state | appliedlastseqno | appliedlatency |
| host1 | extractor | master | ONLINE | 0 | 1.724 |
| host1 | applier | slave | ONLINE | 0 | 0.000 |