From Tungsten Replicator 6.0.4, continuent-tools-hadoop are now packaged within the main Tungsten Replicator software bundle and can be found within ./tungsten-replicator/support/hadoop-tools
The continuent-tools-hadoop
repository contains a set of tools that allow for the convenient
creation of DDL, materialized views, and data comparison on the tables
that have been replicated from MySQL.
To obtain the tools, use git
shell> ./bin/load-reduce-check -s test -Ujdbc:mysql:thin://tr-hadoop2:13306 -udbload -ppasswordThe load-reduce-check command performs four distinct steps:
Reads the schema from the MySQL server and creates the staging table DDL within Hive
Reads the schema from the MySQL server and creates the base table DDL within Hive
Executes the materialized view process on each selected staging table data to build the base table content.
Performs a data comparison