4.4. Upgrading Tungsten Clustering

To upgrade an existing installation on Tungsten Clustering, the new distribution must be downloaded and unpacked, and the included tpm command used to update the installation. The upgrade process implies a small period of downtime for the cluster as the updated versions of the tools are restarted, but downtime is deliberately kept to a minimum, and the cluster should be in the same operation state once the upgrade has finished as it was when the upgrade was started.

4.4.1. Upgrading using the Staging Method (with ssh Access)

To perform an upgrade of an entire cluster from a staging directory installation, where you have ssh access to the other hosts in the cluster:

  1. On your staging server, download the release package.

  2. Unpack the release package:

    shell> tar zxf tungsten-clustering-5.2.0-222.tar.gz
  3. Change to the extracted directory:

    shell> cd tungsten-clustering5.2.0-222
  4. The next step depends on your existing deployment:

    • If you are upgrading a Multi-Site, Multi-Master deployment:

      If you installed the original service by making use of the $CONTINUENT_PROFILES and $REPLICATOR_PROFILES environment variables, no further action needs to be taken to update the configuration information. Confirm that these variables are set before performing the validation and update.

      If you did not use these environment variables when deploying the solution, you must load the existing configuration from the current hosts in the cluster before continuing by using tpm fetch:

      shell> ./tools/tpm fetch --hosts=east1,east2,east3,west1,west2,west3 \
          --user=tungsten --directory=/opt/continuent

      Important

      You must specify ALL the hosts within both clusters within the current deployment when fetching the configuration; use of the autodetect keyword will not collect the correct information.

    • If you are upgrading any other deployment:

      If you are are using the $CONTINUENT_PROFILES variable to specify a location for your configuration, make sure that the variable has been set correctly.

      If you are not using $CONTINUENT_PROFILES, a copy of the existing configuration must be fetched from the installed Tungsten Clustering installation:

      shell> ./tools/tpm fetch --hosts=host1,host2,host3,autodetect \
          --user=tungsten --directory=/opt/continuent

      Important

      You must use the version of tpm from within the staging directory (./tools/tpm) of the new release, not the tpm installed with the current release.

      The current configuration information will be retrieved to be used for the upgrade:

      shell> ./tools/tpm fetch --hosts=host1,host2,host3 --user=tungsten --directory=/opt/continuent
      .......
      NOTE  >> Configuration loaded from host1,host2,host3
  5. Check that the update configuration matches what you expect by using tpm reverse:

    shell> ./tools/tpm reverse
    # Options for the dsone data service
    tools/tpm configure dsone \
    --application-password=password \
    --application-port=3306 \
    --application-user=app_user \
    --connectors=host1,host2,host3 \
    --datasource-log-directory=/var/log/mysql \
    --install-directory=/opt/continuent \
    --master=host1 \
    --members=host1,host2,host3 \
    '--profile-script=~/.bashrc' \
    --replication-password=password \
    --replication-port=13306 \
    --replication-user=tungsten \
    --start-and-report=true \
    --user=tungsten \
    --witnesses=192.168.0.1
  6. Run the upgrade process:

    shell> ./tools/tpm update

    Note

    During the update process, tpm may report errors or warnings that were not previously reported as problems. This is due to new features or functionality in different MySQL releases and Tungsten Clustering updates. These issues should be addressed and the tpm update command re-executed.

    The following additional options are available when updating:

    • --no-connectors (optional)

      By default, an update process will restart all services, including the connector. Adding this option prevents the connectors from being restarted. If this option is used, the connectors must be manually updated to the new version during a quieter period. This can be achieved by running on each host the command:

      shell> tpm promote-connector

      This will result in a short period of downtime (couple of seconds) only on the host concerned, while the other connectors in your configuration keep running. During the upgrade, the Connector is restarted using the updated software and/or configuration.

    A successful update will report the cluster status as determined from each host in the cluster:

    ...........................................................................................................
    Getting cluster status on host1
    Tungsten Clustering for MySQL 5.2.0 build 222
    connect to 'dsone@host1'
    dsone: session established
    [LOGICAL] /dsone > ls
    
    COORDINATOR[host3:AUTOMATIC:ONLINE]
    
    ROUTERS:
    +----------------------------------------------------------------------------+
    |connector@host1[31613](ONLINE, created=0, active=0)                      |
    |connector@host2[27649](ONLINE, created=0, active=0)                      |
    |connector@host3[21475](ONLINE, created=0, active=0)                      |
    +----------------------------------------------------------------------------+
    
    ...
    
    #####################################################################
    # Next Steps
    #####################################################################
    We have added Tungsten environment variables to ~/.bashrc.
    Run `source ~/.bashrc` to rebuild your environment.
    
    Once your services start successfully you may begin to use the cluster.
    To look at services and perform administration, run the following command
    from any database server.
    
      $CONTINUENT_ROOT/tungsten/tungsten-manager/bin/cctrl
    
    Configuration is now complete.  For further information, please consult
    Tungsten documentation, which is available at docs.continuent.com.
    
    NOTE  >> Command successfully completed

The update process should now be complete. The current version can be confirmed by starting cctrl.

4.4.2. Upgrading when using INI-based configuration, or without ssh Access

To perform an upgrade of an individual node, tpm can be used on the individual host. The same method can be used to upgrade an entire cluster without requiring tpm to have ssh access to the other hosts in the dataservice.

Important

Application traffic to the nodes will be disconnected when the connector restarts. Use the --no-connectors tpm option when you upgrade to prevent the connectors from restarting until later when you want them to.

To upgrade a cluster using this method there are two methods, switch and no-switch. The switch method employs a manual cluster failover to a new master, the no-switch method does not. Which method you choose depends upon your needs.

To use the no-switch method of upgrading:

  1. Place the cluster into maintenance mode

  2. Upgrade the slaves in the dataservice. Be sure to shun and welcome each slave.

  3. Upgrade the master node

    Important

    Replication traffic to the slaves will be delayed while the replicator restarts. The delays will increase if there are a large number of stored events in the THL. Old THL may be removed to decrease the delay. Do NOT delete THL that has not been received on all slave nodes or events will be lost.

  4. Upgrade the connectors in the dataservice one-by-one

    Important

    Application traffic to the nodes will be disconnected when the connector restarts.

  5. Place the cluster into automatic mode

To use the switch method of upgrading:

  1. Upgrade the slaves in the dataservice. Be sure to shun and welcome each slave.

  2. Switch the current master to one of the upgraded slaves

    Important

    Application and replication traffic will be delayed while the switch occurs.

  3. Upgrade the original master node which is now a slave. Be sure to shun and welcome it.

  4. Upgrade the connectors in the dataservice one-by-one

    Important

    Application traffic to the nodes will be disconnected when the connector restarts.

For more information on performing maintenance across a cluster, see Section 5.14.3, “Performing Maintenance on an Entire Dataservice”.

To upgrade a single host with tpm:

  1. Download the release package.

  2. Unpack the release package:

    shell> tar zxf tungsten-clustering-5.2.0-222.tar.gz
  3. Change to the extracted directory:

    shell> cd tungsten-clustering-5.2.0-222
  4. Execute tpm update, specifying the installation directory. This will update only this host:

    shell> ./tools/tpm update --directory=/opt/continuent
    NOTE  >> Configuration loaded from host1
    ..............................
    Getting cluster status on host1
    Tungsten Clustering for MySQL 5.2.0 build 222
    connect to 'dsone@host1'
    dsone: session established
    [LOGICAL] /dsone > ls
    
    COORDINATOR[host2:AUTOMATIC:ONLINE]
    ...
    
    #####################################################################
    # Next Steps
    #####################################################################
    We have added Tungsten environment variables to ~/.bashrc.
    Run `source ~/.bashrc` to rebuild your environment.
    
    Once your services start successfully you may begin to use the cluster.
    To look at services and perform administration, run the following command
    from any database server.
    
      $CONTINUENT_ROOT/tungsten/tungsten-manager/bin/cctrl
    
    Configuration is now complete.  For further information, please consult
    Tungsten documentation, which is available at docs.continuent.com.
    
    NOTE  >> Command successfully completed

To update all of the nodes within a cluster, the steps above will need to be performed individually on each host.