In the event of a failure, or during the process of performing maintenance on a running cluster, the roles of the Extractor and Appliers within the cluster may need to be swapped.
The basic sequence of operation for switching Primary and Replicas is:
Switch Replicas to offline state
Switch Primary to offline status
Set an existing Replica to have the
master role
Set each Replica with the slave
role, updating the Extractor URI (where the THL logs will be loaded) to
the new Extractor host
Switch the new $ctpri; to online state
Switch the new $ctreps; to online state
Depending on the situation when the switch is performed, the switch can be performed either without waiting for the hosts to be synchronized (i.e. in a failure situation), or by explicitly waiting for the Replica that will be promoted to the Primary role.
To perform an ordered switch of the Primary. In the example below, Primary
host host1 will be switched to
host3, and the remaining hosts
(host1 and
host2) will be configured as Replicas
to the new Primary:
If you are performing the switch as part of maintenance or other procedures, you should perform a safe switch, ensuring the Replicas are up to date with the Primary:
Synchronize the database and the transaction history log. This will ensure that the two are synchronized, and provide you with a sequence number to ensure the Replicas are up to date:
shell> trepctl -host host1 flush
Master log is synchronized with database at log sequence number: 1405Keep a note of the sequence number.
For each current Replica within the cluster, wait until the Primary sequence number has been reached, and then put the Replica into the offline state:
shell>trepctl -host host2 wait -applied 1405shell>trepctl -host host2 offlineshell>trepctl -host host3 wait -applied 1405shell>trepctl -host host3 offline
If the Primary has failed, or once the Replicas and Primaries are in sync, you can perform the remainder of the steps to execute the physical switch.
Switch the Primary to the offline state:
shell> trepctl -host host1 offline
Configure the new designated Primary to the
Primary role:
shell> trepctl -host host3 setrole -role masterSwitch the Primary to the online state:
shell> trepctl -host host3 onlineFor each Replica, set the role to Replica, supplying the URI of the THL service on the Primary:
shell> trepctl -host host1 setrole -role slave -uri thl://host3:2112In the above example we are using the default THL port (2112).
Put the new Replica into the online state:
shell> trepctl -host host1 onlineRepeat for the remaining Replicas:
shell>trepctl -host host2 setrole -role slave -uri thl://host3:2112shell>trepctl -host host2 online
Once completed, the state of each host can be checked to confirm that the switchover has completed successfully:
appliedLastEventId : mysql-bin.000005:0000000000002100;0 appliedLastSeqno : 1405 appliedLatency : 0.094 dataServerHost : host1 masterConnectUri : thl://host3:2112 role : slave state : ONLINE ----- appliedLastEventId : mysql-bin.000005:0000000000002100;0 appliedLastSeqno : 1405 appliedLatency : 0.149 dataServerHost : host2 masterConnectUri : thl://host3:2112 role : slave state : ONLINE ----- appliedLastEventId : mysql-bin.000005:0000000000002100;0 appliedLastSeqno : 1405 appliedLatency : 0.061 dataServerHost : host3 masterConnectUri : thl://host1:2112/ role : master state : ONLINE
In the above, host1 and
host2 are now getting the THL
information from host1, with each
acting as a Replica to the host1 as
Primary.