6.3.4. Specifying Required Latency

Depending on the selected routing method, load balancer and QoS setting, a slave will automatically be chosen when the host connects. The maximum allowed latency can be set to limit the connection to only use a slave that is within the specified maximum applied latency limit.

This can be specified in the connection string, and enables slave selection based on the slave which has a latency within the specified limit. For example, using the connection string:

jdbc://connector1:3306/database?maxAppliedLatency=5

Will specify that a host with a replication latency of less than 5 seconds should be selected.

The option can be set globally by configuring the JDBC options used by the connector via the --connector-max-slave-latency option to tpm (in seconds):

shell> ./tools/tpm update alpha --connector-max-slave-latency=10

Warning

The Connector computes latency by polling the Replicator every 3 seconds for the current replication-view latency.

This gives the Connector an accuracy of +/- 3 seconds, which means that values of 3 or less will not function as expected.

For any queries that have a very low tolerance for replication latency, we strongly suggest you read directly from the master database server only. This ensures the latest data is being read.

Important

The --connector-max-slave-latency flag does not ensure the slave has applied the latest sequence number, just that its latency at the last commit was under the specified number. This behavior can be adjusted by specifying --use-relative-latency=true in the configuration.

NOTE: --use-relative-latency=true is a cluster-wide setting, cctrl and trepctl will also report relative latency based on this setting.

6.3.4.1. Applied and Relative Latency Comparison

  • Applied Latency

    The appliedLatency is the latency between the commit time of the source event and the time the last committed transaction reached the end of the corresponding pipeline within the replicator. Within a master, this indicates the latency between the transaction commit time and when it was written to the THL. In a slave, it indicates the latency between the commit time on the master database and when the transaction has been committed to the destination database. Clocks must be synchronized across hosts for this information to be accurate. appliedLatency : 0.828 The latency is measured in seconds. Increasing latency may indicate that the destination database is unable to keep up with the transactions from the master. In replicators that are operating with parallel apply, appliedLatency indicates the latency of the trailing channel. Because the parallel apply mechanism does not update all channels simultaneously, the figure shown may trail significantly from the actual latency.

    See Section C.2.6, “Terminology: Fields appliedLatency for more information.

  • Relative Latency

    The relativeLatency is the latency between now and timestamp of the last event written into the local THL. This information gives an indication of how fresh the incoming THL information is. On a master, it indicates whether the master is keeping up with transactions generated on the master database. On a slave, it indicates how up to date the THL read from the master is. A large value can either indicate that the database is not busy, that a large transaction is currently being read from the source database, or from the master replicator, or that the replicator has stalled for some reason. An increasing relativeLatency on the slave may indicate that the replicator may have stalled and stopped applying changes to the dataserver.

    See Section C.2.70, “Terminology: Fields relativeLatency for more information.

  • Comparing Relative and Applied Latencies

    Both relative and applied latency are visible via the trepctl status. Relative indicates the latency since the last time the appliedLastSeqno advanced; for example:

    shell> trepctl status
    
    Processing status command...
    NAME                     VALUE
    ----                     -----
    appliedLastEventId     : mysql-bin.000211:0000000020094766;0
    appliedLastSeqno       : 78022
    appliedLatency         : 0.571
    ...
    relativeLatency        : 8.944
    Finished status command...

    In this example the last transaction had an applied latency (time to write to the slave DB) of 0.571 seconds from the time it committed on the master, and last committed something to the slave DB 8.944 seconds ago.

    If relative latency increases significantly in a busy system, it may be a sign that replication is stalled. This is a good parameter to check in monitoring scripts.

  • For more information,see:

    --connector-max-slave-latency

    --use-relative-latency

    Section 4.1.7.3, “Relative Latency”

6.3.4.2. Advanced Troubleshooting for Latency-based Routing

To troubleshoot the latency-based routing decisions the connector makes, and uncomment the below lines in /opt/continuent/tungsten/tungsten-connector/conf/log4j.properties

#log4j.logger.com.continuent.tungsten.router.resource.loadbalancer=debug, stdout
#log4j.additivity.com.continuent.tungsten.router.resource.loadbalancer=false

The log will then show and explain what node the connector choose and why.

Important

No connector restart is required to enable logging. If you re-comment the lines, a restart will be required. To disable without a connector restart, replace the word debug with the word info.