Tungsten Cluster can operate using either absolute or relative latency. The two are distinguished according to how the difference between transaction commit times are handled:
Absolute latency — (default) is the difference between when a transaction was applied to a Replica and when the transaction was originally applied to the Primary.
Relative latency — is the difference between now and when the last transaction was written to the Replica.
Absolute latency indicates the difference between transaction times, but, may also provide a misleading impression of the cluster state if there are large transactions being applied, or if the Replica has stopped or become 'stuck' due to a transient failure. This is because absolute latency shows the time difference between transactions. If a transaction takes 5 or 10 seconds to apply, the absolute latency will only display the difference between when the transaction was written, and only after this has occurred on both the Primary and the Replica. The actual time difference between these may be less than a second, even though the transaction took 10 seconds to succeed.
Relative latency shows the time difference between the last transaction committed and the current time, hence if the transaction takes a considerable time to be applied, the relative latency will increase up until the transaction has finally been committed. If the relative latency increases and continues to increase, it may indicate a lagging or even failed Replica.
To enable relative latency, the cluster must have been deployed, or
updated, using the
--use-relative-latency=true option to
tpm. Once enabled, the following operational activities
change:
The output of show slave
status when connected to MySQL through a connector will be
updated so that the
Seconds_Behind_Master field
shows the relative, rather than absolute, latency. For example, in a
cluster where relative latency is enabled, but no transactions are
occurring, the output will show an increasing value:
mysql>show slave status\G*************************** 1. row *************************** ... Seconds_Behind_Master: 0 ... 1 row in set (0.01 sec) mysql>show slave status\G*************************** 1. row *************************** ... Seconds_Behind_Master: 7 ... 1 row in set (0.01 sec) mysql>show slave status\G*************************** 1. row *************************** ... Seconds_Behind_Master: 38 ... 1 row in set (0.01 sec)
cctrl will output an additional field,
relative, showing the relative
latency value against the standard latency value. This can be seen in
the example below:
[LOGICAL] /alpha > ls
COORDINATOR[host1:AUTOMATIC:ONLINE]
ROUTERS:
+----------------------------------------------------------------------------+
|connector@host1[6189](ONLINE, created=1, active=0) |
|connector@host2[14253](ONLINE, created=3, active=2) |
|connector@host3[2419](ONLINE, created=1, active=0) |
+----------------------------------------------------------------------------+
DATASOURCES:
+----------------------------------------------------------------------------+
|host1(master:ONLINE, progress=5, THL latency=1.008, relative=144.636) |
|STATUS [OK] [2014/09/07 02:28:44 PM BST] |
+----------------------------------------------------------------------------+
| MANAGER(state=ONLINE) |
| REPLICATOR(role=master, state=ONLINE) |
| DATASERVER(state=ONLINE) |
| CONNECTIONS(created=4, active=1) |
+----------------------------------------------------------------------------+
+----------------------------------------------------------------------------+
|host2(slave:ONLINE, progress=5, latency=0.000, relative=144.638) |
|STATUS [OK] [2014/09/07 02:28:58 PM BST] |
+----------------------------------------------------------------------------+
| MANAGER(state=ONLINE) |
| REPLICATOR(role=slave, master=host1, state=ONLINE) |
| DATASERVER(state=ONLINE) |
| CONNECTIONS(created=1, active=1) |
+----------------------------------------------------------------------------+
+----------------------------------------------------------------------------+
|host3(slave:ONLINE, progress=5, latency=5.938, relative=144.620) |
|STATUS [OK] [2014/09/07 02:29:13 PM BST] |
+----------------------------------------------------------------------------+
| MANAGER(state=ONLINE) |
| REPLICATOR(role=slave, master=host1, state=ONLINE) |
| DATASERVER(state=ONLINE) |
| CONNECTIONS(created=0, active=0) |
+----------------------------------------------------------------------------+
The Tungsten Connector will use the value when the
maxAppliedLatency
option is used in the connection string to determine whether to route
a connection to a Primary or a Replica.
For example, when running a scrip that sends a heartbeat, and then connects through a connector, the connection will be routed first to the Replica, and then to the Primary:
echo "cluster heartbeat" | cctrl sleep 1 mysql -utungsten_testing -pprivate --port=9999 --host=`hostname` \ mysql@maxAppliedLatency=20?qos=RO_RELAXED -e"select 1;tungsten connection status;" sleep 21 mysql -utungsten_testing -pprivate --port=9999 --host=`hostname` \ mysql@maxAppliedLatency=20?qos=RO_RELAXED -e"select 1;tungsten connection status;"
The output of the execution of the script shows the Replica and then Primary connections:
[LOGICAL] /alpha > cluster heartbeat
HEARTBEAT 'DEFAULT' INSERTED
[LOGICAL] /alpha >
Exiting...
+---+
| 1 |
+---+
| 1 |
+---+
+---------------------------------------------------------------------------------+
| Message |
+---------------------------------------------------------------------------------+
| host1@alpha(slave:ONLINE) STATUS(OK), QOS=RO_RELAXED SSL.IN=false SSL.OUT=false |
+---------------------------------------------------------------------------------+
+---+
| 1 |
+---+
| 1 |
+---+
+---------------------------------------------------------------------------------+
| Message |
+---------------------------------------------------------------------------------+
| host1@alpha(master:ONLINE) STATUS(OK), QOS=RO_RELAXED SSL.IN=false SSL.OUT=false |
+---------------------------------------------------------------------------------+