JDunphy-CompileZimbra
Zimbra 9.0P38 Network to Zimbra 10.0P5 FOSS
Example of how to migrate from a Zimbra 9 patch 38 Network version to a Zimbra 10 patch 5 FOSS version. The following shows an in place migration. Prior to running this, we have performed this action:
# su - zimbra % zmcontrol stop
While this isn't necessary, it does allow one to validate that should a license have expired that we can convert our system to the FOSS and continue operation.
This is the first of 3 wiki articles. The next will be FOSS to FOSS installation and the last will cover how to do upgrades.
WARNING: Attempting this without a backup or snapshot could result in loss of data and server.
There is now a script to help build Zimbra that can automate many of the parts listed here.
Summary of Steps
- get environment setup
- compile Zimbra 10 FOSS version
- install Zimbra 10 FOSS version
- verify everything is running
Step 1: Build Environment
There are 2 ways I have tested to compile a version of zimbra. The simplest method is to use Ian's zimbra-build-helper.sh. This helper script will establish your build environment on any of the supported platforms that Zimbra has been ported. The other method will use the zm-build steps documented in the zimbra github. The big difference between the two methods when this article was written is that the zm-build method requires you install the supporting packages manually for your OS and know the tags for the release you would like to build against. Ian's zimbra-build-helper.sh script will build the latest from the development branch but that may change as another tool zimbra-tag-helper has recently surfaced by Adrian Gibanel Lopez to discover the tags for a build. Lastly, double check if you have any filters to prevent Zimbra's FOSS repository from being used. For example, with RHEL environments, it is common to do something like this:
% grep exclude /etc/yum.conf exclude=zimbra
Verify this is commented out prior to installing your release in step 3 below. Afterwords, you can put it back after you have a working system.
% mkdir build-zimbra % cd build-zimbra % git clone https://github.com/ianw1974/zimbra-build-scripts % ls zimbra-build-scripts % zimbra-build-scripts/zimbra-build-helper.sh --help Zimbra Build Helper script! Valid parameters are as follows: --install-deps - Installs required dependencies --build-zimbra - Builds Zimbra --cleanup - Cleanup previous attempted builds --help - Shows this help screen ...
For this example, we will setup the evironment for RHEL 8 (Oracle 8 installation). Run it as a regular user and it will prompt you for sudo password once it starts to install your development environment.
% zimbra-build-scripts/zimbra-build-helper.sh --install-deps
I am going to use the zmbuild method here next to build the release. Here is my script to show this:
% cd build-zimbra % cat mybuild.sh #!/bin/sh # # Note: prerequisite. You must have previously added the ssh keys for the account you run this script as in github. # ref: https://docs.github.com/en/authentication/connecting-to-github-with-ssh/adding-a-new-ssh-key-to-your-github-account export PATH=/usr/local/bin:/usr/sbin:/usr/bin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/home/jad/bin:/usr/sbin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin mkdir installer-build cd installer-build git clone --depth 1 --branch 10.0.5 git@github.com:Zimbra/zm-build.git cd zm-build ENV_CACHE_CLEAR_FLAG=true ./build.pl --ant-options -DskipTests=true --git-default-tag=10.0.5,10.0.4,10.0.3,10.0.2,10.0.1,10.0.0-GA,10.0.0 --build-release-no=10.0.0 --build-type=FOSS --build-rel ease=LIBERTY --build-release-candidate=GA --build-thirdparty-server=files.zimbra.com --no-interactive
Step 2: Build the release
% ./mybuild.sh ... ... ... service httpd stop 2>/dev/null service nginx restart service nginx status EOM_SCRIPT
Step 3: Install the release over the current release
One issue that came up during installation was a port 53 conflict because my server is running bind 9. Follow along in the transcripts to see how to continue and disable this service. At present there is no command line option to not install this service.
% cd installer-build/BUILDS/RHEL8_64-LIBERTY-1000-20240202120614-FOSS-1000/ % ls archive-access-r8.txt archives/ zcs-10.0.0_GA_1000.RHEL8_64.20240202120614.tgz % tar zxvf zcs-10.0.0_GA_1000.RHEL8_64.20240202120614.tgz % ls archive-access-r8.txt archives/ zcs-10.0.0_GA_1000.RHEL8_64.20240202120614/ zcs-10.0.0_GA_1000.RHEL8_64.20240202120614.tgz % cd zcs-10.0.0_GA_1000.RHEL8_64.20240202120614/ % su - # cd installer-build/
Now start the build. Follow along with the transcripts below:
# ./install.sh --skip-ng-check
Operations logged to /tmp/install.log.IOMz2i25
Checking for existing installation...
zimbra-license-tools...FOUND zimbra-license-tools-9.0.0.1701325158-1
zimbra-license-extension...FOUND zimbra-license-extension-9.0.0.1587049946-1
zimbra-network-store...FOUND zimbra-network-store-9.0.0.1587050469-1
zimbra-ldap...FOUND zimbra-ldap-9.0.0_GA_3954
zimbra-logger...FOUND zimbra-logger-9.0.0_GA_3954
zimbra-mta...FOUND zimbra-mta-9.0.0_GA_3954
zimbra-dnscache...FOUND zimbra-dnscache-9.0.0_GA_3954
zimbra-snmp...FOUND zimbra-snmp-9.0.0_GA_3954
zimbra-store...FOUND zimbra-store-9.0.0_GA_3954
zimbra-apache...FOUND zimbra-apache-9.0.0_GA_3954
zimbra-spell...FOUND zimbra-spell-9.0.0_GA_3954
zimbra-convertd...FOUND zimbra-convertd-9.0.0_GA_3954
zimbra-memcached...FOUND zimbra-memcached-1.6.5-1zimbra8.7b1
zimbra-proxy...FOUND zimbra-proxy-9.0.0_GA_3954
zimbra-archiving...NOT FOUND
zimbra-onlyoffice...NOT FOUND
zimbra-core...FOUND zimbra-core-9.0.0_GA_3954
zimbra-zimlet-chat-video-classic...NOT FOUND
zimbra-zimlet-chat-video-modern...NOT FOUND
zimbra-extension-chat-video...NOT FOUND
ZCS upgrade from 9.0.0 to 10.0.0 will be performed.
Warning: You are about to upgrade from the Network Edition to the
Open Source Edition. This will remove all Network features, including
Attachment Searching, Zimbra Mobile, Backup/Restore, and support for the
Zimbra Connector for Outlook.
Do you wish to continue? [N] Y
Validating ldap configuration
LDAP validation succeeded. Continuing.
----------------------------------------------------------------------
PLEASE READ THIS AGREEMENT CAREFULLY BEFORE USING THE SOFTWARE.
SYNACOR, INC. ("SYNACOR") WILL ONLY LICENSE THIS SOFTWARE TO YOU IF YOU
FIRST ACCEPT THE TERMS OF THIS AGREEMENT. BY DOWNLOADING OR INSTALLING
THE SOFTWARE, OR USING THE PRODUCT, YOU ARE CONSENTING TO BE BOUND BY
THIS AGREEMENT. IF YOU DO NOT AGREE TO ALL OF THE TERMS OF THIS
AGREEMENT, THEN DO NOT DOWNLOAD, INSTALL OR USE THE PRODUCT.
License Terms for this Zimbra Collaboration Suite Software:
https://www.zimbra.com/license/zimbra-public-eula-2-6.html
----------------------------------------------------------------------
Do you agree with the terms of the software license agreement? [N] Y
Checking current number of databases...
Do you want to verify message store database integrity? [Y]
Verifying integrity of message store databases. This may take a while.
Starting mysqld...done.
mysqld is alive
No errors found
mysqld is alive
Stopping mysqld... done.
Use Zimbra's package repository [Y]
Configuring package repository
Checking for installable packages
Found zimbra-core (local)
Found zimbra-ldap (local)
Found zimbra-logger (local)
Found zimbra-mta (local)
Found zimbra-dnscache (local)
Found zimbra-snmp (local)
Found zimbra-store (local)
Found zimbra-apache (local)
Found zimbra-spell (local)
Found zimbra-memcached (repo)
Found zimbra-proxy (local)
The Zimbra Collaboration Server appears to already be installed.
It can be upgraded with no effect on existing accounts,
or the current installation can be completely removed prior
to installation for a clean install.
Do you wish to upgrade? [Y]
Scanning for any new or additional packages available for installation
Existing packages will be upgraded
Upgrading zimbra-core
Upgrading zimbra-ldap
Upgrading zimbra-logger
Upgrading zimbra-mta
Upgrading zimbra-dnscache
Upgrading zimbra-snmp
Upgrading zimbra-store
Upgrading zimbra-apache
Upgrading zimbra-spell
Upgrading zimbra-memcached
Upgrading zimbra-proxy
Checking required space for zimbra-core
Checking space for zimbra-store
Checking required packages for zimbra-store
zimbra-store package check complete.
Installing:
zimbra-core
zimbra-ldap
zimbra-logger
zimbra-mta
zimbra-dnscache
zimbra-snmp
zimbra-store
zimbra-apache
zimbra-spell
zimbra-memcached
zimbra-proxy
The system will be modified. Continue? [N] Y
Beginning Installation - see /tmp/install.log.IOMz2i25 for details...
zimbra-core-components will be downloaded and installed.
zimbra-common-core-jar will be installed.
zimbra-common-core-libs will be installed.
zimbra-common-mbox-conf will be installed.
zimbra-common-mbox-conf-attrs will be installed.
zimbra-common-mbox-conf-msgs will be installed.
zimbra-common-mbox-conf-rights will be installed.
zimbra-common-mbox-db will be installed.
zimbra-common-mbox-docs will be installed.
zimbra-common-mbox-native-lib will be installed.
zimbra-timezone-data will be installed.
zimbra-core will be installed.
zimbra-ldap-components will be downloaded and installed.
zimbra-ldap will be installed.
zimbra-logger will be installed.
zimbra-mta-components will be downloaded and installed.
zimbra-mta will be installed.
zimbra-dnscache-components will be downloaded and installed.
zimbra-dnscache will be installed.
zimbra-snmp-components will be downloaded and installed.
zimbra-snmp will be installed.
Last metadata expiration check: 0:01:47 ago on Fri 02 Feb 2024 12:51:39 PM PST.
zimbra-jetty-distribution will be downloaded and installed.
zimbra-store-components will be downloaded and installed.
zimbra-mbox-admin-console-war will be installed.
zimbra-mbox-conf will be installed.
zimbra-mbox-war will be installed.
zimbra-mbox-service will be installed.
zimbra-mbox-store-libs will be installed.
zimbra-mbox-webclient-war will be installed.
zimbra-store will be installed.
zimbra-apache-components will be downloaded and installed.
zimbra-apache will be installed.
zimbra-spell-components will be downloaded and installed.
zimbra-spell will be installed.
Last metadata expiration check: 0:02:07 ago on Fri 02 Feb 2024 12:51:39 PM PST.
zimbra-memcached will be downloaded and installed.
zimbra-proxy-components will be downloaded and installed.
zimbra-proxy will be installed.
Downloading packages (11):
zimbra-core-components
zimbra-ldap-components
zimbra-mta-components
zimbra-dnscache-components
zimbra-snmp-components
zimbra-jetty-distribution
zimbra-store-components
zimbra-apache-components
zimbra-spell-components
zimbra-memcached
zimbra-proxy-components
...done
Saving existing configuration file to /opt/zimbra/.saveconfig
Shutting down zimbra mail
Backing up the ldap database...done.
Removing stale package zimbra-zco while upgrade...done
Upgrading the remote packages
Remove zimbra-connect if it is installed ...
Remove zimbra-connect-modern if it is installed ...
Remove zimbra-drive if it is installed ...
Remove zimbra-drive-ng if it is installed ...
Remove zimbra-drive-modern if it is installed ...
Remove zimbra-docs if it is installed ...
Remove zimbra-docs-modern if it is installed ...
Remove zimbra-chat if it is installed ...
Remove zimbra-talk if it is installed ...
Remove zimbra-zimlet-auth if it is installed ...
Remove zimbra-zimlet-briefcase-edit-lool if it is installed ...
Remove zimbra-network-modules-ng if it is installed ...
zimbra-network-modules-ng FOUND...
Removing zimbra-network-modules-ng...done
Remove zimbra-zimlet-restore-contacts if it is installed ...
zimbra-zimlet-restore-contacts FOUND...
Removing zimbra-zimlet-restore-contacts...done
Remove zimbra-zimlet-duplicate-contacts if it is installed ...
Removing deployed webapp directories
Removing zimbra-modern-ui...done
Installing repo packages (11):
zimbra-core-components
zimbra-ldap-components
zimbra-mta-components
zimbra-dnscache-components
zimbra-snmp-components
zimbra-jetty-distribution
zimbra-store-components
zimbra-apache-components
zimbra-spell-components
zimbra-memcached
zimbra-proxy-components
...done
Installing local packages (26):
zimbra-common-core-jar
zimbra-common-core-libs
zimbra-common-mbox-conf
zimbra-common-mbox-conf-attrs
zimbra-common-mbox-conf-msgs
zimbra-common-mbox-conf-rights
zimbra-common-mbox-db
zimbra-common-mbox-docs
zimbra-common-mbox-native-lib
zimbra-timezone-data
zimbra-core
zimbra-ldap
zimbra-logger
zimbra-mta
zimbra-dnscache
zimbra-snmp
zimbra-mbox-admin-console-war
zimbra-mbox-conf
zimbra-mbox-war
zimbra-mbox-service
zimbra-mbox-store-libs
zimbra-mbox-webclient-war
zimbra-store
zimbra-apache
zimbra-spell
zimbra-proxy
...done
Running Post Installation Configuration:
Setting defaults from saved config in /opt/zimbra/.saveconfig/config.save
HOSTNAME=mail.example.com
LDAPHOST=mail.example.com
LDAPPORT=389
SNMPTRAPHOST=mail.example.com
SMTPSOURCE=admin@example.com
SMTPDEST=admin@example.com
SNMPNOTIFY=yes
SMTPNOTIFY=yes
LDAPROOTPW=*
LDAPZIMBRAPW=*
LDAPPOSTPW=*
LDAPREPPW=*
LDAPAMAVISPW=*
LDAPNGINXPW=*
Restoring existing configuration file from /opt/zimbra/.saveconfig/localconfig.xml...done
Operations logged to /tmp/zmsetup.20240202-125507.log
Adding /opt/zimbra/conf/ca/ca.pem to cacerts
Upgrading from 9.0.0_GA_3954 to 10.0.0_GA_1000
Stopping zimbra services...done.
This appears to be 9.0.0_GA
Starting mysql...done.
Checking ldap status...not running.
Checking ldap status...not running.
Starting ldap...done.
Running mysql_upgrade...done.
Schema upgrade required from version 111 to 118.
Running /opt/zimbra/libexec/scripts/migrate20210506-BriefcaseApi.pl
Fri Feb 2 12:55:43 2024: Verified schema version 111.
Fri Feb 2 12:55:48 2024: Verified schema version 111.
Fri Feb 2 12:55:48 2024: Updating DB schema version from 111 to 112.
Running /opt/zimbra/libexec/scripts/migrate20200625-MobileDevices.pl
Fri Feb 2 12:55:52 2024: Verified schema version 112.
Fri Feb 2 12:55:52 2024: Adding mobile_operator column to ZIMBRA.MOBILE_DEVICES table.
Fri Feb 2 12:55:53 2024: Verified schema version 112.
Fri Feb 2 12:55:53 2024: Updating DB schema version from 112 to 113.
Running /opt/zimbra/libexec/scripts/migrate20210319-MobileDevices.pl
Fri Feb 2 12:55:57 2024: Verified schema version 113.
Fri Feb 2 12:55:57 2024: Adding last_updated_by column to ZIMBRA.MOBILE_DEVICES table.
Fri Feb 2 12:55:59 2024: Verified schema version 113.
Fri Feb 2 12:55:59 2024: Updating DB schema version from 113 to 114.
Running /opt/zimbra/libexec/scripts/migrate20220721-AddMdmUpdateTimestamp.pl
Fri Feb 2 12:56:03 2024: Verified schema version 114.
Fri Feb 2 12:56:03 2024: Adding timestamp column to zimbra.mobile_devices table.
Fri Feb 2 12:56:04 2024: Verified schema version 114.
Fri Feb 2 12:56:04 2024: Updating DB schema version from 114 to 115.
Running /opt/zimbra/libexec/scripts/migrate20220525-Volume.pl
Fri Feb 2 12:56:10 2024: Verified schema version 115.
Fri Feb 2 12:56:10 2024: Adding store_type column to zimbra.volume table.
/opt/zimbra/store
/opt/zimbra/index
Fri Feb 2 12:56:11 2024: Adding store_manager_class column to zimbra.volume table.
Fri Feb 2 12:56:13 2024: Verified schema version 115.
Fri Feb 2 12:56:13 2024: Updating DB schema version from 115 to 116.
Running /opt/zimbra/libexec/scripts/migrate20220729-FilesShareWithMeFolder.pl
Fri Feb 2 12:56:17 2024: Verified schema version 116.
.
Fri Feb 2 12:56:21 2024: Verified schema version 116.
Fri Feb 2 12:56:21 2024: Updating DB schema version from 116 to 117.
Running /opt/zimbra/libexec/scripts/migrate20230224-UpdateOnlyOffice-7.2.1.pl
Fri Feb 2 12:56:25 2024: Verified schema version 117.
Fri Feb 2 12:56:26 2024: Verified schema version 117.
Fri Feb 2 12:56:26 2024: Updating DB schema version from 117 to 118.
Stopping mysql...done.
Updating zimbraLDAPSchemaVersion to version '1673397105'
Updating global config and COS's with attributes introduced after 9.0.0_GA...done.
Stopping ldap...done.
Upgrade complete.
Checking ldap status....not running.
Starting ldap...done.
Setting defaults...done.
Setting defaults from existing config...done.
Checking for port conflicts
Port conflict detected: 53 (zimbra-dnscache)
Port conflict detected: 53 (zimbra-dnscache)
Port conflicts detected! - Press Enter/Return key to continue
Setting defaults from ldap...done.
Main menu
1) Common Configuration:
2) zimbra-ldap: Enabled
3) zimbra-logger: Enabled
4) zimbra-mta: Enabled
5) zimbra-dnscache: Enabled
******* +Master DNS IP address(es): UNSET
+Enable DNS lookups over TCP: yes
+Enable DNS lookups over UDP: yes
+Only allow TCP to communicate with Master DNS: no
6) zimbra-snmp: Enabled
7) zimbra-store: Enabled
8) zimbra-spell: Enabled
9) zimbra-convertd: Enabled
10) zimbra-proxy: Enabled
11) Default Class of Service Configuration:
s) Save config to file
x) Expand menu
q) Quit
Address unconfigured (**) items (? - help) 5
DNS Cache configuration
1) Status: Enabled
** 2) Master DNS IP address(es): UNSET
3) Enable DNS lookups over TCP: yes
4) Enable DNS lookups over UDP: yes
5) Only allow TCP to communicate with Master DNS: no
Select, or 'r' for previous menu [r] 1
DNS Cache configuration
1) Status: Disabled
Select, or 'r' for previous menu [r]
Main menu
1) Common Configuration:
2) zimbra-ldap: Enabled
3) zimbra-logger: Enabled
4) zimbra-mta: Enabled
5) zimbra-dnscache: Disabled
6) zimbra-snmp: Enabled
7) zimbra-store: Enabled
8) zimbra-spell: Enabled
9) zimbra-convertd: Enabled
10) zimbra-proxy: Enabled
11) Default Class of Service Configuration:
s) Save config to file
x) Expand menu
q) Quit
*** CONFIGURATION COMPLETE - press 'a' to apply
Select from menu, or press 'a' to apply config (? - help) a
Saving config in /opt/zimbra/config.1078055...done.
Operations logged to /tmp/zmsetup.20240202-125507.log
Setting local config values...done.
Initializing core config...Setting up CA...done.
Deploying CA to /opt/zimbra/conf/ca ...done.
Setting replication password...done.
Setting Postfix password...done.
Setting amavis password...done.
Setting nginx password...done.
Creating server entry for mail.example.com...already exists.
Setting Zimbra IP Mode...done.
Saving CA in ldap...done.
Saving SSL Certificate in ldap...done.
Setting spell check URL...done.
Setting service ports on mail.example.com...done.
Setting Keyboard Shortcut Preferences...done.
Setting zimbraFeatureTasksEnabled=TRUE...done.
Setting zimbraFeatureBriefcasesEnabled=TRUE...done.
Setting TimeZone Preference...done.
Initializing mta config...done.
Setting services on mail.example.com...done.
Adding mail.example.com to zimbraMailHostPool in default COS...done.
Creating user spam.noqw5vymf_@example.com...already exists.
Creating user ham.yzwm90xvzp@example.com...already exists.
Creating user virus-quarantine.uq078bld@example.com...already exists.
Setting spam training and Anti-virus quarantine accounts...done.
Configuring SNMP...done.
Setting up syslog.conf...done.
Removing convertd mime tree from ldap...done.
Removing convertd from zimbraServiceEnabled list...done.
Removing network components from ldap...done.
Setting zimbraReverseProxySSLProtocols...done.
Setting zimbraReverseProxySSLCiphers...done.
Setting mailboxd_java_options...done.
Setting zimbra_zmjava_options...done.
Setting ldap_common_tlsciphersuite...done.
Setting ldap_common_tlsprotocolmin...done.
Setting amavis_sslversion...done.
Setting java options...done.
Starting servers...done.
Enabling jetty logging...done.
Checking for deprecated zimlets...done.
Checking for network zimlets in LDAP...done.
Removing network zimlets...
Removing com_zimbra_smime_cert_admin...done.
Removing com_zimbra_backuprestore...done.
Removing com_zimbra_hsm...done.
Removing com_zimbra_license...done.
Removing com_zimbra_two_factor_auth...done.
Removing com_zimbra_convertd...done.
Removing com_zimbra_mobilesync...done.
Finished removing network zimlets.
Installing common zimlets...
com_zimbra_ymemoticons...done.
com_zimbra_cert_manager...done.
com_zimbra_phone...done.
com_zimbra_email...done.
com_zimbra_mailarchive...done.
com_zimbra_webex...done.
com_zimbra_date...done.
com_zimbra_tooltip...done.
com_zimbra_proxy_config...done.
com_zimbra_url...done.
com_zimbra_attachcontacts...done.
com_zimbra_viewmail...done.
com_zimbra_attachmail...done.
com_zimbra_bulkprovision...done.
com_zimbra_srchhighlighter...done.
com_zimbra_adminversioncheck...done.
Finished installing common zimlets.
Installing network zimlets...
com_zimbra_archive...done.
Finished installing network zimlets.
Getting list of all zimlets...done.
Updating non-standard zimlets...
Finished updating non-standard zimlets.
Restarting mailboxd...done.
Skipping creation of default domain GAL sync account - existing install detected.
You have the option of notifying Zimbra of your installation.
This helps us to track the uptake of the Zimbra Collaboration Server.
The only information that will be transmitted is:
The VERSION of zcs installed (10.0.0_GA_1000_RHEL8_64)
The ADMIN EMAIL ADDRESS created (admin@example.com)
Notify Zimbra of your installation? [Yes] No
Notification skipped
Setting up zimbra crontab...done.
Moving /tmp/zmsetup.20240202-125507.log to /opt/zimbra/log
Configuration complete - press return to exit
Step 4: Verify Everything is running
# su - zimbra % zmcontrol status Host mail.example.com amavis Running antispam Running antivirus Running convertd Stopped ldap Running logger Running mailbox Running memcached Running mta Running opendkim Running proxy Running service webapp Running snmp Running spell Running stats Running zimbra webapp Running zimbraAdmin webapp Running zimlet webapp Running zmconfigd Running [zimbra@relay4 ~]$ zmcontrol -v Release 10.0.0_GA_1000.RHEL8_64_20240202120614 RHEL8_64 FOSS edition.
Note: convertd has stopped because /opt/zimbra/bin/zmconvertctl is missing. Copy this bash script from another host to /opt/zimbra/bin and do this:
% zmconvertctl start
You are done and have a working Zimbra 10 FOSS edition. A lot of work has been accomplished by Synacor in the past few years to make this possible and a huge effort by Ian Walker to create a builder helper to make setting up your environment very easily. He also is providing FOSS builds for all platforms here: https://techfiles.online/zimbra/.
Limitations
Patches and upgrades are a repeat of this process of building and installing at present. We will continue to use the NETWORK version because we like the features and convience of tested updates but it's nice to know that your data is not locked with the vendor as shown by the transcripts in this wiki article.
More articles written by me, https://wiki.zimbra.com/wiki/JDunphy-Notes
