Recovering from upgrade failure
Recovering from upgrade failure due to remote package installation issues
Coming from an ZCS 8.6 or earlier to ZCS 8.7.6 upgrade? Please scroll down here
With ZCS 8.7 and later, various packages are installed via a remote repository. If installation of these packages fail during the upgrade process, the system can end up in an irrecoverable state unless the proper steps are followed.
During the installation process, an install log is written to /tmp. The exact name of the log file is displayed. For example:
Local packages zimbra-core zimbra-ldap zimbra-logger zimbra-mta zimbra-dnscache zimbra-snmp zimbra-store zimbra-apache zimbra-spell zimbra-convertd zimbra-proxy zimbra-archiving selected for installation Monitor /tmp/install.log.9eeEI4KR for package installation progress Remote package installation started Installing zimbra-core-components zimbra-ldap-components zimbra-mta-components zimbra-dnscache-components zimbra-snmp-components zimbra-store-components zimbra-apache-components zimbra-spell-components zimbra-memcached zimbra-proxy-components....
So in this case, if things break, we would examine /tmp/install.log.9eeEI4KR for the reason as to why it failed. It will be necessary to resolve the logged issue before proceeding. Once the issue is resolved, the upgrade process can be restarted using special options passed to the installer:
./install.sh --force-upgrade --skip-activation-check --skip-upgrade-check
When installing, package selection will have to be manually chosen, as the information about what used to be installed will have been lost. Note that this is logged as part of the original install process so it is easy to recover. From the above example, we see:
Installing zimbra-core-components zimbra-ldap-components zimbra-mta-components zimbra-dnscache-components zimbra-snmp-components zimbra-store-components zimbra-apache-components zimbra-spell-components zimbra-memcached zimbra-proxy-components.... Local packages zimbra-core zimbra-ldap zimbra-logger zimbra-mta zimbra-dnscache zimbra-snmp zimbra-store zimbra-apache zimbra-spell zimbra-convertd zimbra-proxy zimbra-archiving
It is possible then to put together what packages are required to recover.
zimbra-memcached from REMOTE components means zimbra-memcached was installed zimbra-core can be ignored, it is always included zimbra-ldap from LOCAL components means zimbra-ldap was installed zimbra-logger from LOCAL components means zimbra-logger was installed zimbra-mta from LOCAL components means zimbra-mta was installed zimbra-dnscache from LOCAL components means zimbra-dnscache was installed zibmra-snmp from LOCAL components means zimbra-snmp was installed zimbra-store from LOCAL components means zimbra-store was installed zimbra-spell from LOCAL components means zimbra-spell was installed. This will also include zimbra-apache automatically zimbra-convertd from LOCAL components means zimbra-convertd was installed. This will also include zimbra-apache automatically zimbra-proxy from LOCAL components means zimbra-proxy was installed. zimbra-archiving from LOCAL compoents means zimbra-archiving was installed. zimbra-apache from LOCAL components is picked up automatically if zimbra-spell for zimbra-convertd was installed, and is not uniquely selectable
When the menu system prompts for packages to be installed, correctly match the above list to the package installation questions.
CentOS/RHEL 6/7 upgrade ZCS 8.6 or earlier to ZCS 8.7.6
We are aware of an issue with memcached while doing and upgrade from ZCS 8.6.0 or earlier to ZCS 8.7.6 using CentOS/RHEL 6, the affected build has the next md5 and sha256:
OS CENTOS/RHEL6 md5 817039ea8ffd6dc7c051b74afbb32830 OS CENTOS/RHEL7 md5 4aff3d435ecddd72f84dee9c2e896c38 NE CENTOS/RHEL6 md5 c1bce132cdb211a285520512b3082ad2 NE CENTOS/RHEL7 md5 400ba1b1b2b357d789b7dd4263fd2a95
If you have downloaded the build with that md5 and sha256, please delete it before run the upgrade and download the new one from our Downloads page and make sure the md5 and sha256 are the next:
OS CENTOS/RHEL6 md5 57feb6203b8040b4e3a381774095eada OS CENTOS/RHEL7 md5 dd6e71321e1d292443a3b57f5f40ccb6 NE CENTOS/RHEL6 md5 01c34f9b85e361978e544420d983b569 NE CENTOS/RHEL7 md5 8461196343a1a8e2ba12b23686065b0e
Log error and workaround
During the upgrade process the log will show the next error:
Thu Apr 6 15:05:34 2017 *** Running as zimbra user: /opt/zimbra/bin/zmcontrol start Host server.zimbra.com Starting ldap...Done. Starting zmconfigd...Done. Starting dnscache...Done. Starting logger...Done. Starting convertd...Done. Starting mailbox...Done. Starting memcached...Failed. Error: memcached not installed Starting proxy...Done. Starting amavis...Done. Starting antispam...Done. Starting antivirus...Done. Starting opendkim...Done. Starting snmp...Done. Starting spell...Done. Starting mta...Done. Starting stats...Done. Starting service webapp...Done. Starting zimbra webapp...Done. Starting zimbraAdmin webapp...Done. Starting zimlet webapp...Done.
It's important to not run the upgrade process again and just run the next command to install memcached:
yum install zimbra-memcached
If you have run the upgrade process one more time, please open a ticket in the Support Portal. As usual, we strongly recommend to you to have a complete Backup or snapshot before doing any upgrade.