In the event the Active site goes down, and a graceful manual switch is not possible, the composite Active role can be failed over to the Passive cluster using cctrl. The failover command performs the forced switch operation. It will try to update the configuration of the east data service but will not fail if not successful.
In this example, hosts db1 (the composite Primary), db2 and db3 in cluster
east have been shut down. To force dataservice
west to become the primary, login to
a node in that cluster and get into cctrl:
shell>cctrl -multiTungsten Cluster 6.1.25 build 6 west: session established [LOGICAL] / >use global[LOGICAL] /global >lsCOORDINATOR[db4:AUTOMATIC:ONLINE] DATASOURCES: +----------------------------------------------------------------------------+ |east(composite master:SHUNNED(FAILSAFE_SHUN)) | |STATUS [SHUNNED] [2015/04/14 01:46:59 AM UTC] | +----------------------------------------------------------------------------+ +----------------------------------------------------------------------------+ |west(composite slave:ONLINE) | |STATUS [OK] [2015/04/14 01:46:59 AM UTC] | +----------------------------------------------------------------------------+
Mark the east data service as failed
to prevent further actions:
[LOGICAL] /global >datasource east failWARNING: This is an expert-level command: Incorrect use may cause data corruption or make the cluster unavailable. Do you want to continue? (y/n)> y WARNING: UNABLE TO REACH PHYSICAL DATA SERVICE 'east' AT THIS TIME. EXCEPTION: Unable to unable to continue with command because no manager is available in service 'east'. CONTINUING WITH COMMAND COMPOSITE DATA SOURCE 'east' IS NOW IN THE FAILED STATE [LOGICAL] /global >lsCOORDINATOR[db4:AUTOMATIC:ONLINE] DATASOURCES: +----------------------------------------------------------------------------+ |east(composite master:FAILED(MANUALLY-FAILED)) | |STATUS [CRITICAL] [2015/04/14 03:11:27 PM UTC] | |REASON[MANUALLY-FAILED] | +----------------------------------------------------------------------------+ +----------------------------------------------------------------------------+ |west(composite slave:ONLINE) | |STATUS [OK] [2015/04/14 02:37:32 PM UTC] | +----------------------------------------------------------------------------+
Issue the failover command to force
the west dataservice to become the
composite Primary:
[LOGICAL] /global >failoverWARNING: DATA SERVICE 'east' IS NOT AVAILABLE. CANNOT GET STATE WARNING: CAN'T GET POLICY MODE FOR SERVICE 'east'. CONTINUING. WARNING: CAN'T SET POLICY MODE 'maintenance' FOR SERVICE 'east'. CONTINUING. SELECTED SLAVE: 'west@global' WARNING: UNABLE TO REACH PHYSICAL DATA SERVICE 'east' AT THIS TIME. EXCEPTION: Unable to unable to continue with command because no manager is available in service 'east'. CONTINUING WITH COMMAND ENSURING THAT WE CATCH UP WITH THE MOST ADVANCED RELAY composite data source 'west@global' is now OFFLINE WARNING: UNABLE TO REACH PHYSICAL DATA SERVICE 'west' AT THIS TIME. EXCEPTION: Unable to unable to continue with command because no manager is available in service 'east'. CONTINUING WITH COMMAND PUT THE NEW MASTER 'west@global' ONLINE WARNING: CAN'T SET POLICY MODE 'AUTOMATIC' FOR SERVICE 'east'. CONTINUING. REVERT POLICY: MAINTENANCE => AUTOMATIC FAILOVER TO 'west@global' WAS SUCCESSFUL [LOGICAL] /global >lsCOORDINATOR[db4:AUTOMATIC:ONLINE] DATASOURCES: +----------------------------------------------------------------------------+ |east(composite master:SHUNNED(MANUAL-FAILOVER)) | |STATUS [SHUNNED] [2015/04/14 02:13:18 AM UTC] | +----------------------------------------------------------------------------+ +----------------------------------------------------------------------------+ |west(composite master:ONLINE) | |STATUS [OK] [2015/04/14 02:13:23 AM UTC] | +----------------------------------------------------------------------------+
Composite Active Dataservice (Primary) -
west
[LOGICAL] /global >use west[LOGICAL] /west >lsCOORDINATOR[db4:AUTOMATIC:ONLINE] ROUTERS: +----------------------------------------------------------------------------+ |connector@db4[9757](ONLINE, created=0, active=0) | |connector@db5[9781](ONLINE, created=0, active=0) | |connector@db6[9944](ONLINE, created=0, active=0) | +----------------------------------------------------------------------------+ DATASOURCES: +----------------------------------------------------------------------------+ |db4(master:ONLINE, progress=7, THL latency=0.110) | |STATUS [OK] [2015/04/14 02:13:23 AM UTC] | +----------------------------------------------------------------------------+ | MANAGER(state=ONLINE) | | REPLICATOR(role=master, state=ONLINE) | | DATASERVER(state=ONLINE) | | CONNECTIONS(created=0, active=0) | +----------------------------------------------------------------------------+ +----------------------------------------------------------------------------+ |db5(slave:ONLINE, progress=7, latency=0.172) | |STATUS [OK] [2015/04/14 02:13:23 AM UTC] | +----------------------------------------------------------------------------+ | MANAGER(state=ONLINE) | | REPLICATOR(role=slave, master=db4, state=ONLINE) | | DATASERVER(state=ONLINE) | | CONNECTIONS(created=0, active=0) | +----------------------------------------------------------------------------+ +----------------------------------------------------------------------------+ |db6(slave:ONLINE, progress=7, latency=0.173) | |STATUS [OK] [2015/04/14 02:13:23 AM UTC] | +----------------------------------------------------------------------------+ | MANAGER(state=ONLINE) | | REPLICATOR(role=slave, master=db4, state=ONLINE) | | DATASERVER(state=ONLINE) | | CONNECTIONS(created=0, active=0) | +----------------------------------------------------------------------------+