Continuent Documentation

Page in Other Manuals
Parent Sections

C.2.27. Missing events, or events not extracted correctly

Last Updated: 2016-04-20

Condition or Error

You have missing events, or events that have been correctly extraction from a Tungsten Cluster host.


  • There are multiple potential causes for this issue, including, but not limited t


  • Note

    Before proceeding, ensure you have access to the failed master server and the new master server, if one was promoted. Ensure that your environment is properly initialized. For more information, see Initializing shell environment for VMware Continuent (2105429).

    The example operations below are for a service called east.


    To identify transactions that have not been extracted from a MySQL master running VMware Continuent replication:

    1. Identify the last transaction replicated to the slave servers.

      • If a new master has already been promoted or created:

        1. Run the following command:

          shell> trepctl -service east status

          Note the value of latestEpochNumber. Assume 5385 is the value of latestEpochNumber.

        2. Confirm this is the point where replication switched from the failed master to the new master.

          Now determine the THL and associated data from that THL sequence number using:

          shell> thl -service east list -headers -low 5385 -high 5385

          Now compare the last value on each line to ensure they are different. This value indicates the master host for the sequence number.

          If there are no differences, you need to refer to the trepsvc.log file of the new master to find the point where it became the master. Search for [binlog-to-q-0] Setting extractor position from the end of the log. This shows where the master started the extraction process from a new sequence number. Run the above check for each sequence number until you find the point where extraction switched from one master to another.

      • If no new master has been promoted but the master did fail:

        Run the following command to determine the current sequence number on the slaves:

        shell> dsctl -service east get

        If these values are different between multiple slaves, use the lowest reported sequence to reset the position. If the dsctl utility is not available, see the trep_commit_seqno table for information on the last replicated transaction for the slave. The location of this table is different depending on your slave DBMS.

        Assume 5384 is the value of seqno returned by the command.

    2. Check the output of the following command on the master and slaves:

      shell> thl -service east list -headers -seqno 5384

      The output should be used to confirm that each host reports the same information. If there are differences, it is indicative of larger problems and you should reprovision from a known good datasource.

    3. Run the following command to read the events from the master:

      shell> tungsten_read_master_events --service=east --after=5384

      This will display the mysqlbinlog output for any transactions that were applied after the listed sequence number.

      Use this information to determine the next course of action for your application. You may can choose to ignore the transactions if they are not important or add them to the remaining servers manually and then repair replication.

More Information

Section 5.8.1, “Identifying a Transaction Mismatch”