Skip to main content
Common Reference

Generating Advanced Diagnostic Information

To aid in the diagnosis of difficult issues, below are tools and procedures to assist in the data collection.

warning

ONLY execute the below commands and procedures when requested by Continuent support staff.

  • Replicator Thread Dump Procedure

    This procedure creates a replicator memory thread dump for detailed analysis.

    Run this command on replicator hosts when requested by Continuent support.

    This will append the detailed thread dump information to the log file named trepsvc.log in the /opt/continuent/tungsten/tungsten-replicator/log directory.

    shell> su - tungsten
    shell> replicator dump
    shell> tungsten_send_diag -f /opt/continuent/tungsten/tungsten-replicator/log/trepsvc.log -c {case_number}

  • Manager Memory Usage Script

    Applies to: Tungsten Clustering for MySQL

    We have provided a script to easily tell us how much memory a given manager is consuming.

    Place the script on all of your manager hosts (i.e. into the tungsten OS user home directory).

    Note

    The script assumes that cctrl is in the path. If not, then change the script to provide a full path for cctrl.

    shell> su - tungsten
    shell> vi tungsten_manager_memory
    #!/bin/bash
    memval=`echo gc | cctrl | grep used | tail -1 | awk -F: '{print $2}' | tr -d ' |'`
    megabytes=`expr $memval / 1000000`
    timestamp=`date +"%F %T" | tr '-' '/'`
    echo "$timestamp | `hostname` | $megabytes MB"

    shell> chmod 750 tungsten_manager_memory
    shell> ./tungsten_manager_memory

    This script is ideally run from cron and the output redirected to time-stamped log files for later correlation with manager issues.

  • Manager Thread Dump Procedure

    Applies to: Tungsten Clustering for MySQL

    This procedure creates a Manager memory thread dump for detailed analysis.

    Run this command on manager hosts when requested by Continuent support.

    This will append the detailed thread dump information to the log file named tmsvc.log in the /opt/continuent/tungsten/tungsten-manager/log directory.

    shell> su - tungsten
    shell> manager dump
    shell> tungsten_send_diag -f /opt/continuent/tungsten/tungsten-manager/log/tmsvc.log -c {case_number}

  • Manager Heap Dump Procedure

    Applies to: Tungsten Clustering for MySQL

    This procedure creates a Manager memory heap dump for detailed analysis.

    Run this command on manager hosts when requested by Continuent support.

    This will create a file named [hostname].hprof in the directory where you run it.

    shell> su - tungsten
    shell> jmap -dump:format=b,file=`hostname`.hprof `ps aux | grep JANINO | grep -v grep | awk '{print $2}'`
    shell> tungsten_send_diag -f `hostname`.hprof -c {case_number}

  • Configuring Connector Debug Logging

    Applies to: Tungsten Clustering for MySQL

    This procedure allows the Connector to be configured for debug logging.

    Perform this procedure on Connector hosts when requested by Continuent support.

    Warning

    Enabling Connector debug logging will decrease performance dramatically. Disk writes will increase as will disk space consumption. Do not use in production environments unless instructed to do so by Continuent support. In any case, run in this mode for as short a period of time as possible - just long enough to gather the needed debug information. After that is done, disable debug logging.

    To enable debug logging, edit the Connector configuration file tungsten-connector/conf/log4j.properties and turn Connector logging from INFO to DEBUG (or to TRACE for verbose logging):

    shell> su - tungsten
    shell> vi /opt/continuent/tungsten/tungsten-connector/conf/log4j.properties
    # Enable debug for the connector only
    logger.Connector.name=com.continuent.tungsten.connector
    # WAS: logger.Connector.level=INFO
    logger.Connector.level=DEBUG
    logger.Connector.additivity=false
    shell> connector reconfigure

    To disable debug logging, edit the Connector configuration file tungsten-connector/conf/log4j.properties and revert the change from DEBUG to INFO.