8.18.2. thl list Command

The list command to the thl command outputs a list of the sequence number information from the THL. By default, the entire THL as stored on disk is output. Command-line options enable you to select individual sequence numbers, sequence number ranges, or all the sequence information from a single file.

thl list
[-seqno # ]
[-low # ] | [-from # ] | [-high # ] | [-to # ]
[-last] [-last #] [-first] [-first #]
[-file filename ] [-no-checksum ] [-sql] [-sizes] [-sizesdetail] [-sizessummary] [-charset] [-headers] [-json] [-specs-] [-charset]

  • -event eventid

    Output THL found that matches the provided eventid. If no exact match found, a message will display details of an approximate match if found. See example below:

    An exact match is found:

    shell> thl list -event mysql-bin.000017:0000000074628349
    - METADATA = [mysql_server_id=1000;mysql_thread_id=62;unsafe_for_block_commit;dbms_type=mysql;tz_aware=true;service=alpha;shard=employees] 
    - TYPE = com.continuent.tungsten.replicator.event.ReplDBMSEvent 
    - OPTIONS = [##charset = ISO8859_1, autocommit = 1, sql_auto_is_null = 0, foreign_key_checks = 1, unique_checks = 1, sql_mode = 'NO_ENGINE_SUBSTITUTION,STRICT_TRANS_TABLES', character_set_client = 8, collation_connection = 8, collation_server = 8] 
    - SCHEMA = employees 
    - SQL(0) = DROP TABLE `salaries` /* generated by server */

    No match found:

    Event not found : Approximative match found between seqno 915 (mysql-bin.000017:0000000074628153;62) and seqno 916 (mysql-bin.000017:0000000074628349;62)

  • -file filename

    Outputs all of the sequence number fragment information from the specified THL file. If the filename has been determined from the thl index command, or by examining the output of other fragments, the file-based output can be used to identify statements or row data within the THL.

  • -charset charset

    Specify the character set to be used to decode the character-based row data embedded within the THL event. Without this option, data is output as a hex value.

  • -hex

    For SQL that may be in different character sets, the information can be optionally output in hex format to determine the contents and context of the statement, even though the statement itself may be unreadable on the command-line.

  • -no-checksum

    Ignores checksums within the THL. In the event of a checksum failure, use of this option will enable checksums to be ignored when the THL is being read.

  • -sql

    Prints only the SQL for the selected sequence range. Use of this option can be useful if you want to extract the SQL and execute it directly by storing or piping the output.

  • -headers

    Generates only the header information for the selected sequence numbers from the THL. For THL that contains a lot of SQL, obtaining the headers can be used to get basic content and context information without having to manually filter out the SQL in each fragment.

    The information is output as a tab-delimited list:

    2047	1412	0	false	2020-05-03 20:58:14.0	mysql-bin.000005:0000000579721045;0	host3		
    2047	1412	1	true	2020-05-03 20:58:14.0	mysql-bin.000005:0000000579721116;0	host3		
    2048	1412	0	false	2020-05-03 20:58:14.0	mysql-bin.000005:0000000580759206;0	host3		
    2048	1412	1	true	2020-05-03 20:58:14.0	mysql-bin.000005:0000000580759277;0	host3		
    2049	1412	0	false	2020-05-03 20:58:16.0	mysql-bin.000005:0000000581791468;0	host3		
    2049	1412	1	true	2020-05-03 20:58:16.0	mysql-bin.000005:0000000581791539;0	host3		
    2050	1412	0	false	2020-05-03 20:58:18.0	mysql-bin.000005:0000000582812644;0	host3

    The format of the fields output is:

    Sequence No | Epoch | Fragment | Last | Fragment | Date/Time | EventID | SourceID | Comments

    For more information on the fields displayed, see Section E.1.1, “THL Format”.

  • -json

    Only valid with the -headers option, the header information is output for the selected sequence numbers from the THL in JSON format. The field contents are identical, with each fragment of each THL sequence being contained in a JSON object, with the output consisting of an array of the these sequence objects. For example:

    [
       {
          "lastFrag" : false,
          "epoch" : 7,
          "seqno" : 320,
          "time" : "2020-05-02 11:41:19.0",
          "frag" : 0,
          "comments" : "",
          "sourceId" : "host1",
          "eventId" : "mysql-bin.000004:0000000244490614;0"
       },
       {
          "lastFrag" : true,
          "epoch" : 7,
          "seqno" : 320,
          "time" : "2020-05-02 11:41:19.0",
          "frag" : 1,
          "comments" : "",
          "sourceId" : "host1",
          "eventId" : "mysql-bin.000004:0000000244490685;0"
       }
    ]

    For more information on the fields displayed, see THL SEQNO.

  • -sizes

    Shows the size information for a given THL event, describing either the size of the SQL, or the number of rows within the given event. For example:

    shell> thl list -sizes
    SEQ#	Frag#	Tstamp
    ...
    12	0	2020-06-28 13:21:11.0		Event total: 1 chunks	73 bytes in SQL statements	0 rows
    13	0	2020-06-28 13:21:10.0		Event total: 1645 chunks	0 bytes in SQL statements	1645 rows
    14	0	2020-06-28 13:21:11.0		Event total: 1 chunks	36 bytes in SQL statements	0 rows
    15	0	2020-06-28 13:21:11.0		Event total: 1 chunks	61 bytes in SQL statements	0 rows
    16	0	2020-06-28 13:21:11.0		Event total: 1 chunks	73 bytes in SQL statements	0 rows
    17	0	2020-06-28 13:21:12.0		Event total: 1 chunks	36 bytes in SQL statements	0 rows
    18	0	2020-06-28 13:21:12.0		Event total: 1 chunks	61 bytes in SQL statements	0 rows
    19	0	2020-06-28 13:21:10.0		Event total: 1784 chunks	0 bytes in SQL statements	1784 rows
    20	0	2020-06-28 13:21:12.0		Event total: 1 chunks	73 bytes in SQL statements	0 rows
    21	0	2020-06-28 13:21:11.0		Event total: 1576 chunks	0 bytes in SQL statements	1576 rows
    22	0	2020-06-28 13:21:12.0		Event total: 1 chunks	36 bytes in SQL statements	0 rows
    23	0	2020-06-28 13:21:12.0		Event total: 1 chunks	61 bytes in SQL statements	0 rows
    ...

    Summary information is also output identicating an overall count of the changes. For example:

    Total ROW chunks: 69487 with 18257671 updated rows (100%)
    Total STATEMENT chunks: 0 with 0 bytes (0%)
    628 events processed

    This information can be useful when viewing or monitoring the replication progress as it can help to indicate and identify the size of a specific transaction, particularly if the transaction is large. This can be particularly useful in combination with the -first and/or -last.

    For more detailed information on individual fragments within a sequence (and for large transactions there will be multiple fragments), use the thl list -sizesdetail command.

  • -sizesdetail

    Shows detailed size information for a given THL event, describing either the size of the SQL, or the number of rows within the given event per fragment within each event, and with a summary for each event total. For very large THL event sizes this provide more detailed information about the size and makeup of the event. For example:

    shell> thl list -sizes -last
    SEQ#	Frag#	Tstamp			Chunks		SQL Data				Row Data
    1604	0	2020-06-29 11:04:53.0	123 chunks	SQL 0 bytes (0 avg bytes per chunk)	Rows 45633 (371 avg rows per chunk)
    1604	1	2020-06-29 11:04:53.0	123 chunks	SQL 0 bytes (0 avg bytes per chunk)	Rows 45633 (371 avg rows per chunk)
    1604	2	2020-06-29 11:04:53.0	123 chunks	SQL 0 bytes (0 avg bytes per chunk)	Rows 45633 (371 avg rows per chunk)
    1604	3	2020-06-29 11:04:53.0	123 chunks	SQL 0 bytes (0 avg bytes per chunk)	Rows 45633 (371 avg rows per chunk)
    1604	4	2020-06-29 11:04:53.0	123 chunks	SQL 0 bytes (0 avg bytes per chunk)	Rows 45633 (371 avg rows per chunk)
    1604	5	2020-06-29 11:04:53.0	123 chunks	SQL 0 bytes (0 avg bytes per chunk)	Rows 45633 (371 avg rows per chunk)
    1604	6	2020-06-29 11:04:53.0	123 chunks	SQL 0 bytes (0 avg bytes per chunk)	Rows 45633 (371 avg rows per chunk)
    1604	7	2020-06-29 11:04:53.0	123 chunks	SQL 0 bytes (0 avg bytes per chunk)	Rows 45633 (371 avg rows per chunk)
    1604	8	2020-06-29 11:04:53.0	123 chunks	SQL 0 bytes (0 avg bytes per chunk)	Rows 45633 (371 avg rows per chunk)
    1604	9	2020-06-29 11:04:53.0	123 chunks	SQL 0 bytes (0 avg bytes per chunk)	Rows 45633 (371 avg rows per chunk)
    1604	10	2020-06-29 11:04:53.0	123 chunks	SQL 0 bytes (0 avg bytes per chunk)	Rows 45633 (371 avg rows per chunk)
    1604	11	2020-06-29 11:04:53.0	7 chunks	SQL 0 bytes (0 avg bytes per chunk)	Rows 2535 (362 avg rows per chunk)
    			Event total: 1360 chunks	0 bytes in SQL statements	504498 rows

    Summary information is also output identicating an overall count of the changes. For example:

    Total ROW chunks: 69487 with 18257671 updated rows (100%)
    Total STATEMENT chunks: 0 with 0 bytes (0%)
    628 events processed

    This information can be useful when viewing or monitoring the replication progress as it can help to indicate and identify the size of a specific transaction, particularly if the transaction is large. This can be particularly useful in combination with the -first and/or -last.

  • -sizessummary

    Outputs only the size summary information for the requested THL:

    shell> thl list -sizessummary
    Total ROW chunks: 69487 with 18257671 updated rows (100%)
    Total STATEMENT chunks: 0 with 0 bytes (0%)
    628 events processed
  • -specs

    Shows the column specifications, such as identified type, length, and additional settings, when viewing events within row-based replication. This can be helpful when examining THL data in heterogeneous replication deployments.

    For example:

    shell> thl list -low 5282 -specs
    SEQ# = 5282 / FRAG# = 0 (last frag)
    - TIME = 2020-01-30 05:46:26.0
    - EPOCH# = 5278
    - EVENTID = mysql-bin.000017:0000000000001117;0
    - SOURCEID = host1
    - METADATA = [mysql_server_id=1687011;dbms_type=mysql;is_metadata=true;»
       service=firstrep;shard=tungsten_firstrep;heartbeat=MASTER_ONLINE]
    - TYPE = com.continuent.tungsten.replicator.event.ReplDBMSEvent
    - SQL(0) =
     - ACTION = UPDATE
     - SCHEMA = tungsten_firstrep
     - TABLE = heartbeat
     - ROW# = 0
      - COL(index=1 name= type=4 [INTEGER] length=8 unsigned=false blob=false desc=null) = 1
      - COL(index=2 name= type=4 [INTEGER] length=8 unsigned=false blob=false desc=null) = 1416
      - COL(index=3 name= type=12 [VARCHAR] length=0 unsigned=false blob=false desc=null) = [B@65b60280
      - COL(index=4 name= type=93 [TIMESTAMP] length=0 unsigned=false blob=false desc=null) = 2020-01-30 05:46:26.0
      - COL(index=5 name= type=93 [TIMESTAMP] length=0 unsigned=false blob=false desc=null) = 2020-05-03 12:05:47.0
      - COL(index=6 name= type=4 [INTEGER] length=8 unsigned=false blob=false desc=null) = 1015
      - COL(index=7 name= type=4 [INTEGER] length=8 unsigned=false blob=false desc=null) = 0
      - COL(index=8 name= type=12 [VARCHAR] length=0 unsigned=false blob=false desc=null) = [B@105e55ab
      - KEY(index=1 name= type=4 [INTEGER] length=8 unsigned=false blob=false desc=null) = 1

    When identifying the different data types, the following effects should be noted:

  • -timezone

    Specify the timezone to use when display date or time values. When not specified, times are displayed using UTC.