Compression occurs "in-flight" and is requested by the client replicator prior to fetching the THL from the remote THL Server.

Enabling THL Compression

The following property should be added to your configuration:

For ini installations:

repl-thl-client-serialization=[SERIALIZATION PROTOCOL]

For example:

repl-thl-client-serialization=PROTOBUF

For Staging installations:

--repl-thl-client-serialization=[SERIALIZATION PROTOCOL]

For example:

--repl-thl-client-serialization=DEFLATE

Valid values for the protocol are: LEGACY, JAVA, PROTOBUF or DEFLATE

The default for this property if not supplied is LEGACY. This retains the behavior in versions prior to v7.0.0 and has the same effect as disabling compression.

DEFLATE offers the highest level of compression, but at the cost of being slower during the compression and decompression stages

JAVA should not be used in production, and is mainly used for testing purposes.

The THL Server can be configured to accept one or more of these protocols. By default, a server will support ALL protocols. This can be adjusted as follows:

repl-thl-server-serialization=[COMMA SEPARATED LIST OF PROTOCOLS]

For example, the following would disable DEFLATE:

repl-thl-server-serialization=LEGACY,PROTOBUF

If a client asks for a protocol that is not enabled, it will fall back to LEGACY

THL Encryption and Compression On-Disk can be enabled at install time or dynamically.

The following property should be added to your configuration:

For ini installations:

replicator-store-thl-encrypted=true|false
replicator-store-thl-compressed=true|false

For Staging installations:

--replicator-store-thl-encrypted=true|false
--replicator-store-thl-compressed=true|false

By default, both encryption and compression are disabled.

To change these settings dynamically, the service will need to be put offline first. This will force the replicator to rotate to a new THL log file which will use the new settings.

The commands to enable or disable these settings are:

shell> trepctl [-service servicename] thl -compression {enable|disable}
shell> trepctl [-service servicename] thl -encryption {enable|disable}

The full steps to enable encryption follows:

shell> trepctl [-service servicename] offline
shell> trepctl [-service servicename] thl -encryption enable
shell> trepctl [-service servicename] online

As a result, after this command, thl index command will show the newly generated THL log file as encrypted:

shell> thl [-service servicename] index
….
LogIndexEntry thl.data.0000000008(42:42)
LogIndexEntry thl.data.0000000009(43:43) - ENCRYPTED (thl.ct1691.16)

Encryption uses dedicated keystore and truststore (named by default tungsten_thl_keystore.jks and tungsten_thl_truststore.ts). Losing these files will make encrypted THL log files impossible to be decoded.

The Replicator can generate new keys to be used. These keys will then be sent through the THL protocol to other nodes.

To generate a new key , the following command needs to be executed while the service is online:

shell> trepctl [-service servicename] thl -encryption genkey

This will result in a new THL log file to be started, using the new generated key, as shown by thl index command:

shell> thl [-service servicename] index
….
LogIndexEntry thl.data.0000000008(42:42)
LogIndexEntry thl.data.0000000009(43:45) - ENCRYPTED (thl.ct1691.16)
LogIndexEntry thl.data.0000000010(46:46) - ENCRYPTED (thl.ct1691.46)