Replicating into Kerberos Secured HDFS

For replicating into HDFS where Kerberos support has been enabled, the hadoop_kerberos.js batch script can be used in place of the normal hadoop.js script.

The script will need modification before it can be used, due to the varying implementations of Kerberos, and to ensure the correct authentication parameters are used.

Before installing, edit the hadoop_kerberos.js file located within tungsten-replicator/appliers/batch/hadoop-kerberos.js within the installation package. Within that file is the line called before the HDFS operations are called:

var kinit_prefix = "kinit USER/LEVEL@REALM -k -t KEYTAB_FILE;"

Edit this line to set the correct command and/or authentication parameters, such as the username and keytab file. The configured command will be executed immediately before all the commands that operate on the Hadoop filesystem, including creating directories and files.

For example, the variable might be updated to:

var kinit_prefix = "kinit mc/admin@CLOUDERA -k -t mcadmin.keytab;"

When installing, use batch-load-template=hadoop_kerberos.js to enable the new batch load script.