The ddlscan Command
The ddlscan command scans the existing schema for a database or table and then generates a schema or file in
a target database environment. For example, ddlscan is used in MySQL to Oracle heterogeneous deployments to
translate the schema definitions within MySQL to the Oracle format. For more information on heterogeneous
deployments, see "Understanding Heterogeneous Deployments".
For example, to generate Oracle DDL from an existing MySQL database:
shell> ddlscan -user tungsten -url 'jdbc:mysql:thin://tr-source:13306/test' -pass password \
-template ddl-mysql-oracle.vm -db test
SQL generated on Thu Sep 11 15:39:06 BST 2019 by ./ddlscan utility of Tungsten
url = jdbc:mysql:thin://tr-source:13306/test
user = tungsten
dbName = test
*/
DROP TABLE test.sales;
CREATE TABLE test.sales
(
id NUMBER(10, 0) NOT NULL,
salesman CHAR,
planet CHAR,
value FLOAT,
PRIMARY KEY (id)
);
ddlscan [options]
Where:
| Option | Description |
|---|---|
-conf | Path to a static-[svc].properties file to read JDBC connection address and credentials. |
-db | Database to use (will substitute ${DBNAME} in the URL, if needed) |
-help | Generates the help text of arguments. |
-opt | Option(s) to pass to template. HINT: try -opt help me |
-out | Render to file (print to stdout if not specified) |
-pass | JDBC Password. |
-path | Add additional search path for loading Velocity templates. |
-rename | Definitions file for renaming schemas, tables and columns. |
-service | Name of a replication service instead of path to config. |
-tableFile | New-line separated definitions file of tables to find. |
-tables | Comma-separated list of tables to find. |
-template | Specify template file to render. |
-url | JDBC connection string (use single quotes to escape) |
-user | JDBC Username. |
ddlscan supports three different methods for execution:
Using an explicit JDBC URL, username and password:
shell> ddlscan -user tungsten -url 'jdbc:mysql:thin://tr-hadoop1:13306/test' -user user \-pass password ...This is useful when a deployment has not already been installed.
By specifying an explicit configuration file:
shell> ddlscan -conf /opt/continuent/tungsten/tungsten-replicator/conf/static-alpha.properties ...When an existing deployment has been installed, by specifying one of the active services:
shell> ddlscan -service alpha ...
In addition, the following two options must be specified on the command-line:
- The template to be used (using the
-templateoption) for the DDL translation must be specified on the command-line. A list of the support templates and their operation are shown below in "Supported Templates and Usage" - The
-dbparameter, which defines the database or schema that should be scanned. All tables are translated unless an explicit list, regex, or table file has been specified.
For example, to translate MySQL DDL to Oracle for all tables within the schema test using the connection to
MySQL defined in the service alpha:
shell> ddlscan -service alpha -template ddl-mysql-oracle.vm -db test
ddlscan Options Reference
The details of all the options available to ddlscan are as follows:
-conf
static-[svc].properties file to read JDBC connection address and credentials.-db
${DBNAME} in the URL, if needed)-help
-opt
shell> ddlscan -service alpha -template ddl-mysql-oracle.vm -db test -opt schemaPrefix mysql_
-out
-pass
-path
-rename
-service
-tableFile
tablelist.txt containing the following:typetwo,Table of type two customer forms
typethree,Table of type three customer forms
ddlscan as follows:shell> ddlscan -service alpha -template ddl-mysql-oracle.vm -db test -tableFile tablelist.txt
-tables
shell> ddlscan -service alpha -template ddl-mysql-oracle.vm -db test -tables typetwo,typethree
-template
-url
-user
Supported Templates and Usage
| Option | Description |
|---|---|
ddl-check-pkeys.vm | Reports which tables are without primary key definitions. |
ddl-mysql-hive-0.10-staging.vm | Generates DDL from a MySQL host suitable for the staging tables in a Hadoop/Hive Environment. |
ddl-mysql-hive-0.10.vm | Generates DDL from a MySQL host suitable for the base tables in a Hadoop/Hive Environment. |
ddl-mysql-hive-metadata.vm | Generates metadata as JSON to be used within a Hadoop/Hive Environment. |
ddl-mysql-oracle-cdc.vm | Generates Oracle tables with CDC capture information from a MySQL schema. |
ddl-mysql-oracle.vm | Generates Oracle schema from a MySQL schema. |
ddl-mysql-postgres-staging.vm | Generates DDL from a MySQL host suitable for the base tables in PostgreSQL. |
ddl-mysql-postgres.vm | Generates DDL from a MySQL host suitable for the base tables in PostgreSQL. |
ddl-mysql-redshift-staging.vm | Generates DDL from a MySQL host suitable for the staging tables in Amazon Redshift. |
ddl-mysql-redshift.vm | Generates DDL from a MySQL host suitable for the base tables in Amazon Redshift. |
ddl-mysql-vertica-staging.vm | Generates DDL suitable for the staging tables in HP Vertica. |
ddl-mysql-vertica.vm | Generates DDL suitable for the base tables in HP Vertica. |
ddl-check-pkeys.vm
ddl-check-pkeys.vm template can be used to check whether specific tables within a schema do not have a primary key:shell> ddlscan -template ddl-check-pkeys.vm -user tungsten -pass password -db sales -url jdbc:mysql://localhost:13306/sales
/*
SQL generated on Thu Sep 04 10:23:52 BST 2014 by ./ddlscan utility of Tungsten
url = jdbc:mysql://localhost:13306/sales
user = tungsten
dbName = sales
*/
/* ERROR: sales.dummy1 has no primary key! *//*
SQL generated on Thu Sep 04 10:23:52 BST 2014 by ./ddlscan utility of Tungsten
url = jdbc:mysql://localhost:13306/sales
user = tungsten
dbName = sales
*/
/* ERROR: sales.dummy2 has no primary key! *//*
SQL generated on Thu Sep 04 10:23:52 BST 2014 by ./ddlscan utility of Tungsten
url = jdbc:mysql://localhost:13306/sales
user = tungsten
dbName = sales
*/
ddl-mysql-hive-0.10-staging.vm
sales in MySQL:mysql> describe sales;
+----------+----------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+----------+----------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| salesman | char(20) | YES | | NULL | |
| planet | char(20) | YES | | NULL | |
| value | float | YES | | NULL | |
+----------+----------+------+-----+---------+----------------+
4 rows in set (0.00 sec)
shell> ddlscan -user tungsten -url 'jdbc:mysql://tr-hadoop1:13306/test' -pass password -template ddl-mysql-hive-0.10-staging.vm -db test
--
-- SQL generated on Thu Sep 11 12:31:45 BST 2014 by Tungsten ddlscan utility
--
-- url = jdbc:mysql://tr-hadoop1:13306/test
-- user = tungsten
-- dbName = test
--
DROP TABLE IF EXISTS test.stage_xxx_sales;
CREATE EXTERNAL TABLE test.stage_xxx_sales
(
tungsten_opcode STRING ,
tungsten_seqno INT ,
tungsten_row_id INT ,
tungsten_commit_timestamp TIMESTAMP ,
id INT,
salesman STRING,
planet STRING,
value DOUBLE )
ROW FORMAT DELIMITED FIELDS TERMINATED BY '\001' ESCAPED BY '\\'
LINES TERMINATED BY '\n'
STORED AS TEXTFILE LOCATION '/user/tungsten/staging/test/sales' ;
ddl-mysql-hive-0.10.vm for more information.ddl-mysql-hive-0.10.vm
shell> ddlscan -user tungsten -url 'jdbc:mysql://tr-hadoop1:13306/test' -pass password -template ddl-mysql-hive-0.10.vm -db test
--
-- SQL generated on Thu Sep 11 12:57:11 BST 2014 by Tungsten ddlscan utility
--
-- url = jdbc:mysql://tr-hadoop1:13306/test
-- user = tungsten
-- dbName = test
--
DROP TABLE IF EXISTS test.sales;
CREATE TABLE test.sales
(
id INT,
salesman STRING,
planet STRING,
value DOUBLE )
;
| MySQL Datatype | Hive Datatype |
|---|---|
DATETIME | STRING |
TIMESTAMP | TIMESTAMP |
DATE | STRING |
YEAR | INT |
TIME | STRING |
TINYINT | TINYINT |
TINYINT UNSIGNED | SMALLINT |
SMALLINT | SMALLINT |
SMALLINT UNSIGNED | INT |
MEDIUMINT | INT |
INT | INT |
INT UNSIGNED | BIGINT |
BIGINT | BIGINT |
BIGINT UNSIGNED | STRING |
DECIMAL | STRING |
VARCHAR | STRING |
CHAR | STRING |
BINARY | BINARY |
VARBINARY | BINARY |
TEXT | STRING |
BLOB | BINARY |
FLOAT | DOUBLE |
DOUBLE | DOUBLE |
ENUM | STRING |
SET | STRING |
BIT | STRING |
-opt schemaPrefixA prefix to be placed in front of all schemas. For example, if called withschemaPrefixset tomysql_:shell> ddlscan ... -opt schemaPrefix mysql_The schema name will be prefixed, translating the schema name fromsalesintomysql_sales.-opt tablePrefixA prefix to be placed in front of all schemas. For example, if called withtablePrefixset tomysql_:shell> ddlscan ... -opt tablePrefix mysql_The table name will be prefixed, translating the tablename fromsalesintomysql_sales.
ddl-mysql-hive-metadata.vm
shell> ddlscan -user tungsten -url 'jdbc:mysql://tr-hadoop1:13306/test' -pass password -template ddl-mysql-hive-metadata.vm -db test
{
"tables": [
{
"schema": "test",
"name": "sales",
"keys": ["id"],
"columns": [
{"name": "id", "type": "INT"},
{"name": "salesman", "type": "STRING"},
{"name": "planet", "type": "STRING"},
{"name": "value", "type": "DOUBLE"}
]
}
]
}
ddl-mysql-oracle-cdc.vm
ddl-mysql-oracle-cdc.vm template generates identical tables in Oracle, from their MySQL equivalent, but with additional columns for CDC capture. For example:shell> ddlscan -user tungsten -url 'jdbc:mysql://tr-hadoop1:13306/test' -pass password -template ddl-mysql-oracle-cdc.vm -db test
/*
SQL generated on Thu Sep 11 13:17:05 BST 2014 by ./ddlscan utility of Tungsten
url = jdbc:mysql://tr-hadoop1:13306/test
user = tungsten
dbName = test
*/
DROP TABLE test.sales;
CREATE TABLE test.sales
(
id NUMBER(10, 0) NOT NULL,
salesman CHAR,
planet CHAR,
value FLOAT,
CDC_OP_TYPE VARCHAR(1), /* CDC column */
CDC_TIMESTAMP TIMESTAMP, /* CDC column */
CDC_SEQUENCE_NUMBER NUMBER PRIMARY KEY /* CDC column */
);
ddl-mysql-oracle.vm
| MySQL Datatype | Oracle Datatype |
|---|---|
INT | NUMBER(10, 0) |
BIGINT | NUMBER(19, 0) |
TINYINT | NUMBER(3, 0) |
SMALLINT | NUMBER(5, 0) |
MEDIUMINT | NUMBER(7, 0) |
DECIMAL(x,y) | NUMBER(x, y) |
FLOAT | FLOAT |
CHAR(n) | CHAR(n) |
VARCHAR(n) | VARCHAR2(n) (n < 2000), CLOB (n > 2000) |
DATE | DATE |
DATETIME | DATE |
TIMESTAMP | DATE |
TEXT | CLOB |
BLOB | BLOB |
ENUM(...) | VARCHAR(255) |
ENUM(...) | VARCHAR(4000) |
BIT(1) | NUMBER(1) |
- Table names are translated to uppercase.
- Column names are translated to uppercase.
- If a column name is a reserved word in Oracle, then the column name has an underscore character appended (for example,
TABLEbecomesTABLE_).
- If the table name starts with a number.
- If the table name exceeds 30 characters in length.
- If the table name is a reserved word in Oracle.
- If the column or column name started with a number.
- If the column name exceeds 30 characters in length, the column name will be truncated.
- If the column name is a reserved word in Oracle.
ddl-mysql-postgres-staging.vm
ddl-mysql-postgres-staging.vm template generates DDL for PostgreSQL staging tables. These include the full table definition, in addition to four columns used to define the staging data, including the operation code, sequence number and unique row ID. For example:shell> ddlscan -user tungsten -url 'jdbc:mysql://tr-pgsql:13306/hr' -pass password -template ddl-mysql-postgres-staging.vm -db hr
/*
SQL generated on Thu Sep 11 14:20:14 BST 2014 by ./ddlscan utility of Tungsten
url = jdbc:mysql://tr-pgsql:13306/hr
user = tungsten
dbName = hr
*/
CREATE SCHEMA hr;
DROP TABLE hr.stage_xxx_countries;
CREATE TABLE hr.stage_xxx_countries
(
tungsten_opcode CHAR(2),
tungsten_seqno BIGINT,
tungsten_row_id BIGINT,
tungsten_commit_timestamp TIMESTAMP,
country_id CHAR(2),
country_name VARCHAR(40),
region_id INT,
PRIMARY KEY (country_id)
);
ddl-mysql-postgres.vm
ddl-mysql-postgres.vm template generates DDL for generating tables within a PostgreSQL database from an existing MySQL database schema. For example:shell> ddlscan -user tungsten -url 'jdbc:mysql://tr-pgsql:13306/hr' -pass password -template ddl-mysql-postgres.vm -db hr
/*
SQL generated on Thu Sep 11 14:20:14 BST 2014 by ./ddlscan utility of Tungsten
url = jdbc:mysql://tr-pgsql:13306/hr
user = tungsten
dbName = hr
*/
CREATE SCHEMA hr;
DROP TABLE hr.countries;
CREATE TABLE hr.countries
(
country_id CHAR(2),
country_name VARCHAR(40),
region_id INT,
PRIMARY KEY (country_id)
);
ddl-mysql-redshift-staging.vm
ddl-mysql-redshift-staging.vm template generates DDL for Amazon Redshift tables from MySQL schemas. For example:CREATE TABLE test.stage_xxx_all_mysql_types
(
tungsten_opcode CHAR(2),
tungsten_seqno INT,
tungsten_row_id INT,
tungsten_commit_timestamp TIMESTAMP,
my_id INT,
my_bit BOOLEAN /* BIT(1) */,
my_tinyint SMALLINT /* TINYINT(4) */,
my_boolean SMALLINT /* TINYINT(1) */,
my_smallint SMALLINT,
my_mediumint INT /* MEDIUMINT(9) */,
my_int INT,
my_bigint BIGINT,
my_decimal_10_5 DECIMAL(10,5),
my_float FLOAT,
my_double DOUBLE PRECISION /* DOUBLE */,
my_date DATE,
my_datetime DATETIME,
my_timestamp TIMESTAMP,
my_time VARCHAR(17) /* WARN: no pure TIME type in Redshift */,
my_year YEAR(4) /* ERROR: unrecognized (type=0, length=0) */,
my_char_10 CHAR(10),
my_varchar_10 VARCHAR(40) /* VARCHAR(10) */,
my_tinytext VARCHAR(65535) /* WARN: MySQL TINYTEXT translated to max VARCHAR */,
my_text VARCHAR(65535) /* WARN: MySQL TEXT translated to max VARCHAR */,
my_mediumtext VARCHAR(65535) /* WARN: MySQL MEDIUMTEXT translated to max VARCHAR */,
my_longtext VARCHAR(65535) /* WARN: MySQL LONGTEXT translated to max VARCHAR */,
my_enum_abc VARCHAR(1) /* ENUM('A','B','C') */,
my_set_def VARCHAR(65535) /* SET('D','E','F') */,
PRIMARY KEY (tungsten_opcode, tungsten_seqno, tungsten_row_id)
);
ddl-mysql-redshift.vm
ddl-mysql-redshift.vm template generates DDL for Amazon Redshift tables from MySQL schemas. For example:CREATE TABLE test.all_mysql_types
(
my_id INT,
my_bit BOOLEAN /* BIT(1) */,
my_tinyint SMALLINT /* TINYINT(4) */,
my_boolean SMALLINT /* TINYINT(1) */,
my_smallint SMALLINT,
my_mediumint INT /* MEDIUMINT(9) */,
my_int INT,
my_bigint BIGINT,
my_decimal_10_5 DECIMAL(10,5),
my_float FLOAT,
my_double DOUBLE PRECISION /* DOUBLE */,
my_date DATE,
my_datetime DATETIME,
my_timestamp TIMESTAMP,
my_time VARCHAR(17) /* WARN: no pure TIME type in Redshift */,
my_year YEAR(4) /* ERROR: unrecognized (type=0, length=0) */,
my_char_10 CHAR(10),
my_varchar_10 VARCHAR(40) /* VARCHAR(10) */,
my_tinytext VARCHAR(65535) /* WARN: MySQL TINYTEXT translated to max VARCHAR */,
my_text VARCHAR(65535) /* WARN: MySQL TEXT translated to max VARCHAR */,
my_mediumtext VARCHAR(65535) /* WARN: MySQL MEDIUMTEXT translated to max VARCHAR */,
my_longtext VARCHAR(65535) /* WARN: MySQL LONGTEXT translated to max VARCHAR */,
my_enum_abc VARCHAR(1) /* ENUM('A','B','C') */,
my_set_def VARCHAR(65535) /* SET('D','E','F') */,
PRIMARY KEY (my_id)
);
| MySQL Datatype | Redshift Datatype |
|---|---|
BIGINT | BIGINT |
BINARY | BINARY, CHAR |
BIT(1) | BOOLEAN |
BIT | CHAR |
BLOB | VARBINARY, VARCHAR |
CHAR | CHAR |
DATE | DATE |
DATETIME | DATETIME |
DECIMAL | DECIMAL |
DOUBLE | DOUBLE PRECISION |
ENUM | VARCHAR |
FLOAT | FLOAT |
INT | INT |
LONGBLOB | VARBINARY, CHAR |
LONGTEXT | VARCHAR |
MEDIUMBLOB | VARBINARY, CHAR |
MEDIUMINT | INT |
MEDIUMTEXT | VARCHAR |
SET | VARCHAR |
SMALLINT | SMALLINT |
TEXT | VARCHAR |
TIME | VARCHAR |
TIMESTAMP | TIMESTAMP |
TINYBLOB | VARBINARY, CHAR |
TINYINT | SMALLINT |
TINYTEXT | VARCHAR |
VARBINARY | VARBINARY, CHAR |
VARCHAR | VARCHAR |
- When translating the DDL for
CHARandVARCHARcolumns, the actual column size is increased by a factor of four. This is because Redshift tables always stored data using 32-bit UTF characters and column sizes are in bytes, not characters. Therefore aCHAR(20)column is created asCHAR(80)within Redshift. TEXTcolumns are converted to a RedshiftVARCHARof 65535 in length (the maximum allowed).BLOBcolumns are converted to a RedshiftVARBINARYof 65000 in length (the maximum allowed).BITcolumns with a size of 1 are converted to RedshiftBOOLEANcolumns, larger sizes are converted toCHARcolumns of 64 bytes in length.TIMEcolumns are converted to a RedshiftVARCHARof 17 bytes in length since no explicitTIMEtype exists.
ddl-mysql-vertica-staging.vm
ddl-mysql-vertica-staging.vm template generates DDL for HP Vertica staging tables. These include the full table definition, in addition to four columns used to define the staging data, including the operation code, sequence number and unique row ID. For example:shell> ddlscan -user tungsten -url 'jdbc:mysql://tr-vert:13306/hr' -pass password -template ddl-mysql-vertica-staging.vm -db hr
/*
SQL generated on Wed Mar 25 14:35:40 UTC 2026 by ./ddlscan utility of Tungsten
url = jdbc:mysql://tr-vert:13306/hr
user = tungsten
dbName = hr
*/
CREATE SCHEMA "hr";
DROP TABLE "hr"."stage_xxx_countries";
CREATE TABLE "hr"."stage_xxx_countries"
(
"tungsten_opcode" CHAR(1) ,
"tungsten_seqno" INT ,
"tungsten_row_id" INT ,
"tungsten_commit_timestamp" TIMESTAMP ,
"country_id" CHAR(2) ,
"country_name" VARCHAR(40) ,
"region_id" INT ) ORDER BY "tungsten_seqno", "tungsten_row_id";
ddl-mysql-vertica.vm
ddl-mysql-vertica.vm template generates DDL for generating tables within an HP Vertica database from an existing MySQL database schema. For example:shell> ddlscan -user tungsten -url 'jdbc:mysql://tr-vert1:13306/hr' -pass password -template ddl-mysql-vertica.vm -db hr
/*
SQL generated on Wed Mar 25 14:39:49 UTC 2026 by ./ddlscan utility of Tungsten
url = jdbc:mysql://tr-vert1:13306/test
user = tungsten
dbName = hr
*/
CREATE SCHEMA "hr";
DROP TABLE "hr"."countries";
CREATE TABLE "hr"."countries"
(
"country_id" CHAR(2) ,
"country_name" VARCHAR(40) ,
"region_id" INT )
ORDER BY "country_id"
;
| MySQL Datatype | Vertica Datatype |
|---|---|
DATETIME | DATETIME |
TIMESTAMP | TIMESTAMP |
DATE | DATE |
TIME | TIME |
TINYINT | TINYINT |
SMALLINT | SMALLINT |
MEDIUMINT | INT |
INT | INT |
BIGINT | INT |
VARCHAR | VARCHAR |
CHAR | CHAR |
BINARY | BINARY |
VARBINARY | VARBINARY |
TEXT, TINYTEXT, MEDIUMTEXT, LONGTEXT | VARCHAR(65000) |
BLOB, TINYBLOB, MEDIUMBLOB, LONGBLOB | VARBINARY(65000) |
FLOAT | FLOAT |
DOUBLE | DOUBLE PRECISION |
ENUM | VARCHAR |
SET | VARCHAR(4000) |
BIT(1) | BOOLEAN |
BIT | CHAR(64) |
DECIMALMySQL type is not supported.TEXTtypes in MySQL are converted to aVARCHARin Vertica of the maximum supported size.BLOBtypes in MySQL are converted to aVARBINARYin Vertica of the maximum supported size.SETtypes in MySQL are converted to aVARCHARin Vertica of 4000 characters, designed to work in tandem with thesettostringfilter.ENUMtypes in MySQL are converted to aVARCHARin Vertica of the size of the longestENUMvalue, designed to work in tandem with theenumtostringfilter.