The Transaction History Log (THL) stores transactional data from different data servers in a universal format that is then used to exchange and transfer the information between replicator instances. Because the THL is stored and independently managed from the data servers that it reads and writes, the data can be moved, exchanged, and transmuted during processing.
The THL is created by any replicator service acting as a Primary, where the information is read from the database using the native format, such as the MySQL binary log, or Oracle Change Data Capture (CDC), writing the information to the THL. Once in the THL, the THL data can be exchanged with other processes, including transmission over the network, and then applied to a destination database. Within Tungsten Replicator, this process is handled through the pipeline stages that read and write information between the THL and internal queues.
Information stored in THL is recorded in a series of event records in sequential format. The THL therefore acts as a queue of the transactions. On a replicator reading data from a database, the THL represents the queue of transactions applied on the source database. On a replicator applying that information to a database, the THL represents the list of the transactions to be written. The THL has the following properties:
THL is a sequential list of events
THL events are written to a THL file through a single thread (to enforce the sequential nature)
THL events can be read from individually or sequentially, and multiple threads can read the same THL at the same time
THL events are immutable; once stored, the contents of the THL are never modified or individually deleted (although entire files may be deleted)
THL is written to disk without any buffering to prevent software failure causing a problem; the operating system buffers are used.
THL data is stored on disk within the thl directory of
your Tungsten Replicator installation. The exact location can configured using
logDir parameter of the THL component. A sample
directory is shown below:
total 710504 -rw-r--r-- 1 tungsten tungsten 0 May 2 10:48 disklog.lck -rw-r--r-- 1 tungsten tungsten 100042900 Jun 4 10:10 thl.data.0000000013 -rw-rw-r-- 1 tungsten tungsten 101025311 Jun 4 11:41 thl.data.0000000014 -rw-rw-r-- 1 tungsten tungsten 100441159 Jun 4 11:43 thl.data.0000000015 -rw-rw-r-- 1 tungsten tungsten 100898492 Jun 4 11:44 thl.data.0000000016 -rw-rw-r-- 1 tungsten tungsten 100305613 Jun 4 11:44 thl.data.0000000017 -rw-rw-r-- 1 tungsten tungsten 100035516 Jun 4 11:44 thl.data.0000000018 -rw-rw-r-- 1 tungsten tungsten 101690969 Jun 4 11:45 thl.data.0000000019 -rw-rw-r-- 1 tungsten tungsten 23086641 Jun 5 21:55 thl.data.0000000020
The THL files have the format thl.data.#########, and
the sequence number increases for each new log file. The size of each log
file is controlled by the
--thl-log-file-size configuration
parameter. The log files are automatically managed by Tungsten Replicator,
with old files automatically removed according to the retention policy set
by the --thl-log-retention configuration
parameter. The files can be manually purged or moved. See
Section D.1.5.1, “Purging THL Log Information on a Replica”.
The THL can be viewed and managed by using the thl command. For more information, see Section 8.19, “The thl Command”.