Building the software yourself: Difference between revisions

(Building 5.0RC1 from source)
m (script typo)
Line 83: Line 83:
         make -C ${P4_DIR}/ThirdParty/tcmalloc
         make -C ${P4_DIR}/ThirdParty/tcmalloc
         mkdir -p ${P4_DIR}/ThirdPartyBuilds/${BUILD_PLATFORM}/nginx/
         mkdir -p ${P4_DIR}/ThirdPartyBuilds/${BUILD_PLATFORM}/nginx/
         make -C ${P4_DIR}/ThirdParty/ngnix
         make -C ${P4_DIR}/ThirdParty/nginx
         make -C ${P4_DIR}/ThirdParty/memcached
         make -C ${P4_DIR}/ThirdParty/memcached
         # Provide expected filenames
         # Provide expected filenames
Line 98: Line 98:
         mkdir -p ${P4_DIR}/ThirdParty/java
         mkdir -p ${P4_DIR}/ThirdParty/java
         cp ${JDK16_DIR}.tgz ${P4_DIR}/ThirdParty/java/jdk1.6.0_02.tgz
         cp ${JDK16_DIR}.tgz ${P4_DIR}/ThirdParty/java/jdk1.6.0_02.tgz
         mkdir -p ${P4_DIR}/ThirdPartyBuilds/${BUILD_PLATFORM}/ngnix
         mkdir -p ${P4_DIR}/ThirdPartyBuilds/${BUILD_PLATFORM}/nginx
         # copy windows binaries (.msi,.exe) or use fake touch
         # copy windows binaries (.msi,.exe) or use fake touch
         mkdir -p ${P4_DIR}/ZimbraServer/src/windows/
         mkdir -p ${P4_DIR}/ZimbraServer/src/windows/
Line 132: Line 132:

You will need to download both JDK (5 and 6) packages to build Zimbra.
You will need to download both JDK (5 and 6) packages to build Zimbra.

== Fedora Core 5 x86_64 Source Build ==
== Fedora Core 5 x86_64 Source Build ==

Revision as of 13:05, 29 October 2007

Building Zimbra 5.0.0_RC1 from sources with ThirdParty 5.0.0_BETA3 packages

It seems that in the process of building "open source" edition and latest changes of abandoning SourceForge repository, building from scratch is not so easy as it seems. The following script helps building from sources available at the moment (September 2007).

# Building Zimbra 5.0.0_RC1 with 5.0.0_BETA3 ThirdParty tools
# Some required packages:
# mysql: g++ libncurses5-dev libncurses5w-dev
# cyrus-sasl : libxml2-dev libcurl4-openssl-dev
# PostFix : libsasl2-dev ?
# dspam : libmysqlclient15-dev
# snmp : libperl-dev
# zimbra : apt-get install ant
set -e

test "$1" = "clean" && rm -rf ${P4_DIR}
test -f ${ZIMBRA_SRC} || wget
test -f ${THIRD_PARTY_SRC} || wget ${SOURCEFORGE_MIRROR}/sourceforge/zimbra/${THIRD_PARTY_SRC}

test -d ${P4_DIR} || tar xvzf ${ZIMBRA_SRC}

if test ! -d ${P4_DIR}/ThirdParty ; then
       tar xvz -C ${P4_DIR} -f ${THIRD_PARTY}.tar.gz
       mv ${P4_DIR}/${THIRD_PARTY} ${P4_DIR}/ThirdParty

if  ! test -f ${JDK15_BIN}  ; then
       echo "Download JDK 5.0 Update 13 from"
       exit 1

if  ! test -f ${JDK16_BIN}  ; then
       echo "Download JDK 6 Update 3"
       exit 1
test -d ${JDK15_DIR} || sh ${JDK15_BIN}

if test ! -d ${JDK16_DIR} ; then
       sh ${JDK16_BIN}
       tar cvzf ${JDK16_DIR}.tgz  ${JDK16_DIR}

export JAVA_HOME=${PWD}/${JDK15_DIR}


# ThirdParty software must be build by root as it uses install target
if test  ! -f  ${P4_DIR}/ThirdParty/.built ; then
       # assure some system required users before build
       grep -q zimbra /etc/group || adduser zimbra --disabled-login --home /opt/zimbra
       if  ! grep -q postfix /etc/group ; then
               addgroup --system postfix
               adduser --system --ingroup postfix --no-create-home postfix
       if  ! grep -q postdrop /etc/group ;then
               addgroup --system postdrop
               adduser --system --ingroup postdrop --no-create-home postdrop
       # wipeout is mandatory due to bad linking
       rm -rf /opt/zimbra
       # typo in Makefile(s) that we correct aferwards
       sed -i.orig -e '/chmod -R a+w/d' \
               ${P4_DIR}/ThirdParty/clamav/Makefile \
       # Build packages as provided by ThirdParty/Makefile
       # All should succeed except some Perl modules
       make -C ${P4_DIR}/ThirdParty
       # Build missing packages
       # if libunwind is unavailable then wget libunwind, ./configure and make install
       make -C ${P4_DIR}/ThirdParty/tcmalloc
       mkdir -p ${P4_DIR}/ThirdPartyBuilds/${BUILD_PLATFORM}/nginx/
       make -C ${P4_DIR}/ThirdParty/nginx
       make -C ${P4_DIR}/ThirdParty/memcached
       # Provide expected filenames
       cp ${P4_DIR}/ThirdPartyBuilds/${BUILD_PLATFORM}/mysql/mysql-standard*.tar.gz \
       mkdir -p ${P4_DIR}/ThirdPartyBuilds/${BUILD_PLATFORM}/perl
       cp ${P4_DIR}/ThirdParty/Perl/builds/${BUILD_PLATFORM}/perllib.tgz \
       ( cd ${P4_DIR}/ThirdParty/amavisd
               rm -rf amavisd-new-2.5.2
               tar xvzf amavisd-new-2.5.2.tar.gz
               patch -p1 -d amavisd-new-2.5.2 < amavisd-2.5.2.patch
       mkdir -p ${P4_DIR}/ThirdParty/java
       cp ${JDK16_DIR}.tgz ${P4_DIR}/ThirdParty/java/jdk1.6.0_02.tgz
       mkdir -p ${P4_DIR}/ThirdPartyBuilds/${BUILD_PLATFORM}/nginx
       # copy windows binaries (.msi,.exe) or use fake touch
       mkdir -p ${P4_DIR}/ZimbraServer/src/windows/
       touch ${P4_DIR}/ZimbraServer/src/windows/ZimbraToast-fake.msi
       chmod -R a+w ${P4_DIR}/ThirdPartyBuilds/*
       chmod -R a+w ${P4_DIR}/ThirdParty/*
       touch ${P4_DIR}/ThirdParty/.built

test -e /usr/local/java || ln -sf $JAVA_HOME /usr/local/java
# Zimbra needs to be compiled with jdk1.5 while it runs with 1.6
make -C ${P4_DIR}/ZimbraBuild allclean all

This script shows building process and required fixes to ThirdParty software that ZCS uses. It is verified to build on Debian and Ubuntu. But getting to the end is not so simple. It assumes that all required development packages are available. Some of them are listed at the top of the script. If some ThirdParty package fails, one should look at /tmp/ThirdParty.make.log to figure out cause of the failing package.

After initial packages build, script additionally builds and adjust reguired software locations. It should be noted that ThirdPary build needs to be run as root as packages are installed and loaded in shared library loader path during build.

The last line builds Zimbra suite. But until ThirdParty is complete, it is recommendet that you comment this line.

Quick instructions to try this approach is to the following as root:

mkdir /tmp/zimbra
cd /tmp/zimbra
cat >
# Paste script here and press ^D
chmod +x
# install development packages somehow (apt-get, aptitude, dselect, ...)

You will need to download both JDK (5 and 6) packages to build Zimbra.

Fedora Core 5 x86_64 Source Build

See Building_Zimbra_on_Fedora_Core_5_and_x86_64.

Ubuntu Breezy Source Build


After trying to build the source myself using readme_source.txt and and build_cvs_howto.txt, I decided there must be a better way. It was quite apparent that the processes for building the Third Party components in those HOWTOs hadn't been followed for quite some time. There were issues with Makefiles and the process was generally unwieldy.

Here is what I did on an Ubuntu 5.10 (Breezy) system. This process was performed on a laptop without DNS and then confirmed on a VMWare Ubuntu instance.

Since Ubuntu is a Debian based system, I downloaded the Debian Developer build from [1].

I successfully installed it using the Ubuntu Breezy Hack posted in the forums ([2]). I managed to get to the Zimbra login and use the system.

Then I went through a couple instances of trying to install the source using readme_source.txt and and build_cvs_howto.txt. As a result of getting that process to work, I ended up with something like the following packages installed. This may or not have a bearing on the process I will describe below.

sudo apt-get install sudo libidn11 curl fetchmail libgmp3c2 libxml2 libstdc++6 openssl
sudo apt-get install libssl-dev
sudo apt-get install ssl-cert
sudo apt-get install zlib1g zlib1g-dev
sudo apt-get install autoconf
sudo apt-get install libpcre3  libpcre3-dev expat libpopt0 libpopt-dev
sudo apt-get install bison flex
sudo apt-get install libtool libltdl3-dev
sudo apt-get install automake1.7 
sudo apt-get install libbz2-dev bzip2 libbz2-1.0
sudo apt-get install libgmp3-dev
sudo apt-get install curl libwww-dev libcurl3-dev
sudo apt-get install libsasl2 libgssapi1-heimdal
sudo apt-get install libsasl2-modules-gssapi-heimdal
sudo apt-get install libxml-dev

(or in one line:)

sudo aptitude install libidn11 curl fetchmail libgmp3c2 libxml2 libstdc++6 openssl libssl-dev ssl-cert zlib1g zlib1g-dev autoconf libpcre3  libpcre3-dev expat libpopt0 libpopt-dev bison flex libtool libltdl3-dev automake1.7 libbz2-dev bzip2 libbz2-1.0 libgmp3-dev curl libwww-dev libcurl3-dev libsasl2 libgssapi1-heimdal libsasl2-modules-gssapi-heimdal libxml-dev

For dapper drake, you need libdb3 if it is not already installed:

sudo apt-get install libdb3

The Process

  1. Download the Debian Developer build from [3].
  2. Unpack the tarball
  3. We'll build a new zimbra_core package for Ubuntu

The developer tarball (in this case zcs-3.1.0_GA_332.DEBIAN3.1.tgz) unpacks into the following tree:

 |-- bin
 |-- data
 |-- docs
 |-- packages
 `-- util
     `-- modules

Assuming you unpack it into the directory ~/repack, you can:

$ mkdir ~/repack
$ tar xfz zcs-3.1.0_GA_332.DEBIAN3.1.tgz
$ cd zcs
$ cd packages
$ mkdir zimbra_core
$ dpkg -x zimbra-core_3.1.0_GA_332.DEBIAN3.1_i386.deb zimbra_core
$ cd zimbra_core
$ dpkg -e ../zimbra-core_3.1.0_GA_332.DEBIAN3.1_i386.deb
$ cd ../..
$ pwd

Now apply Ubuntu patches as described below (I couldn't work out how to up load non-media files, would prefer to host the patches at

The first patch changes the list of prerequisite packages in util/ to include libgmp3c2 and libgmp3-dev instead of libgmp3:

-- util/  2006-04-15 05:39:38.000000000 +1000
+++ util/    2006-04-24 08:55:23.000000000 +1000
@@ -972,7 +972,7 @@
        PACKAGERM='dpkg --purge'
        PACKAGEQUERY='dpkg -s'
-       PREREQ_PACKAGES="sudo libidn11 curl fetchmail libgmp3 libxml2 libstdc++6 openssl"
+       PREREQ_PACKAGES="sudo libidn11 curl fetchmail libgmp3c2 libgmp3-dev libxml2 libstdc++6 openssl"
        PACKAGEINST='rpm -iv'
        PACKAGERM='rpm -ev --noscripts --allmatches'

Apply it with:

$ wget
$ patch -p0 < ubuntu1.patch
patching file util/
$ rm ubuntu1.patch

The second patch does the same to packages/zimbra_core/DEBIAN/control:

--- ./DEBIAN/control    2006-04-15 05:36:24.000000000 +1000
+++ ../zimbra_core_new/./DEBIAN/control 2006-04-24 08:41:30.000000000 +1000
@@ -5,4 +5,4 @@
 Section: Mail
 Priority: optional
 Architecture: i386
-Depends: libidn11,curl,fetchmail,libgmp3
+Depends: libidn11,curl,fetchmail,libgmp3c2,libgmp3-dev

Apply it with:

$ cd packages/zimbra_core
$ wget
$ patch -p0 < ubuntu2.patch 
patching file ./DEBIAN/control
$ rm ubuntu2.patch

Now repackage the patched zimbra-core deb archive:

$ cd ..
$ mv zimbra-core_3.1.0_GA_332.DEBIAN3.1_i386.deb zimbra-core_3.1.0_GA_332.DEBIAN3.1_i386.deb.aside
$ dpkg -b zimbra_core zimbra-core_3.1.0_GA_332.DEBIAN3.1_i386.deb

Now /etc/debian_version needs to be edited to trick bin/ Add a blank line and then 3.1. The script in the forums did this as well.

$ cat /etc/debian_version


cd back up to the top of your zcs tree

$ sudo ./

Notes on installation questions

I did 2 installations. One on a VMWare Ubuntu Breezy VM with static IP and hostname in my local network DNS.

I also did an install where I used the host as localhost.localdomain with no DNS on my laptop. So either I kept or entered (can't remember) localhost.localdomain as the host when it said the host can not be resolved by DNS. In this case you need to run 'zmprov mcf zimbraMtaDnsLookupsEnabled FALSE' as the zimbra user after the installation has completed.

For the MX question I said no both times.

I set the admin password as directed by the script (option 6 then 4 etc)

Then you'll see stuff like the following for an almost successful installation

*** CONFIGURATION COMPLETE - press 'a' to apply
Select from menu, or press 'a' to apply config (? - help) a
Save configuration data to a file? [Yes]
Save config in file: [/opt/zimbra/config.8962]
Saving config in /opt/zimbra/config.8962...Done
The system will be modified - continue? [No] yes
Operations logged to /tmp/zmsetup.log.8962
Setting local config values...Done
Setting up CA...Done
Creating SSL certificate...Done
Initializing ldap...Done
Saving CA in ldap...Done
Installing SSL certificate...Done
Creating server entry for
Setting spell check URL...Done
Setting service ports on
Adding to zimbraMailHostPool in default COS...Done
Installing zimlets... com_zimbra_phone... com_zimbra_amzn... com_zimbra_wikipedia... com_zimbra_tracking... com_zimbra_url... com_zimbra_ymaps... com_zimbra_xslt... Done
Setting MTA auth host...Done
Creating domain
Creating user
Creating user
Setting spam training accounts...Done
Creating user
Creating postmaster alias...Done
Initializing store sql database...Done
Setting zimbraSmtpHostname for
Initializing logger sql database...Done
Initializing mta config...Done
Configuring SNMP...Done
Setting services on
Setting up zimbra crontab...Done

You have the option of notifying Zimbra of your installation.
This helps us to track the uptake of the Zimbra Collaboration Suite.
The only information that will be transmitted is:

The VERSION of zcs installed (3.1.0_GA_332_DEBIAN3.1)

Notify Zimbra of your installation? [Yes]
Notifying Zimbra of installation via

Notification complete 

Starting servers...Done.

Operations logged to /tmp/zmsetup.log.8962

Configuration complete - press return to exit

Browse to see if the Zimbra client works http://parker/

Browse to the Zimbra Administration console at https://parker:7071 or https://parker:7071/zimbraAdmin/ and login with admin@yourdomain and the password you set during installation.

Using the Admin console, you should create some more accounts for testing or add passwords to the accounts created during the installation.

The next thing is being able to deploy source. The ant scripts expect the Tomcat manager webapp to be listening on port 7070.

Now the Tomcat manager setup seemed broken so I modified that and also installed the developer version of the Tomcat server.xml file. This must be installed over since that file is used to overwrite server.xml.

Check the source out from CVS.

cvs -z6 -d co main

I have placed mine in a directory zcs_src

zimbra@parker:~$ mv /opt/zimbra/tomcat/conf/ /opt/zimbra/tomcat/conf/
zimbra@parker:~$ cp ~peter/zimbra/zcs_src/ZimbraServer/conf/tomcat-5.5/server.xml /opt/zimbra/tomcat/conf/
zimbra@parker:~$ cd /opt/zimbra/tomcat/conf/Catalina/localhost/
zimbra@parker:~/tomcat/conf/Catalina/localhost$ cp -p manager.xml manager.xml.aside

Apply the following patch

The patch adds 'debug="0"' and 'path="/manager"' to the <Context> element:

--- manager.xml.broken  2006-04-26 15:08:55.000000000 +1000
+++ manager.xml 2006-04-26 15:10:04.000000000 +1000
@@ -7,8 +7,8 @@

-<Context docBase="${catalina.home}/server/webapps/manager"
-         privileged="true" antiResourceLocking="false" antiJARLocking="false">
+<Context path="/manager" docBase="${catalina.home}/server/webapps/manager"
+      debug="0"   privileged="true" antiResourceLocking="false" antiJARLocking="false">

   <ResourceLink name="users" global="UserDatabase"

Apply it with:

zimbra@parker:~/tomcat/conf/Catalina/localhost$ patch -p0 < ~peter/zimbra/tomcatManager.patch
patching file manager.xml

Logout out as the zimbra user and log back in to pick up the new environment from the installation as Zimbra edit ~/.bashrc

Add these lines to your shell rc file:

zimbra@parker:~$ logout
peter@parker:~$ sudo su - zimbra
zimbra@parker:~$ tomcat restart
$ netstat -nat 

and you should no longer see port 80 being used and port 7070 has a listener now

browse to zimbra


you don't need the domain to login as one of the users created above

Try sending some mail to an account you created or the one you have logged in as.

You'll need to hit "Get Mail" to refresh your Inbox.

Now, let's see if the tomcat manager is working

browse to


the html directory was a surprise to me. Took a while to find that. Login as zimbra with password zimbra.

If there is a login issue you should check /opt/zimbra/tomcat/conf/tomcat-users.zml against zcs_src/ZimbraServer/conf/tomcat-5.5/tomcat-users.xml. The Debian developer build at the moment shipped with the zimbra user set up in the manager role.

Modify some permissions

$ sudo chown peter:peter /opt/zimbra/conf/msgs/
$ chmod 644 /opt/zimbra/conf/msgs/
$ sudo chgrp zimbra /opt/zimbra/apache-tomcat-5.5.15/work/Catalina/localhost/zimbraAdmin/tldCache.ser
peter@parker:~/zimbra/zcs_src/ZimbraServer$ sudo chmod g+w $_
peter@parker:~/zimbra/zcs_src$ cd ZimbraWebClient/
peter@parker:~/zimbra/zcs_src/ZimbraWebClient$ ls
bin  build  build.xml  img  jars  src  WebRoot  ZimbraWebClient.iml
peter@parker:~/zimbra/zcs_src/ZimbraWebClient$ export JAVA_HOME=/opt/zimbra/java
peter@parker:~/zimbra/zcs_src/ZimbraWebClient$ export ANT_HOME=/opt/ant
peter@parker:~/zimbra/zcs_src/ZimbraWebClient$ export PATH=$JAVA_HOME/bin:$ANT_HOME/bin:$PATH

Now see if you can compile

$ ant

OK see if you can deploy

$ ant deploy 

and also check /opt/zimbra/tomcat/logs/catalina.out

Browse to the zimbra app and see if it works.

$ cd ../ZimbraServer/
$ ant
$ ant service-deploy

That does crash Tomcat but you probably managed to deploy the webapp. Tomcat crashes on (see /opt/zimbra/tomcat/log/catalina.out)

Zimbra server reserving server socket port=110 bindaddr=null ssl=false
Fatal error: terminating: PrivilegedServlet init failed Permission denied
       at Method)
       at com.zimbra.cs.util.NetUtil.newBoundServerSocket(
       at com.zimbra.cs.util.NetUtil.reserveServerSocket(
       at com.zimbra.cs.servlet.PrivilegedServlet.init(
       at javax.servlet.GenericServlet.init(
       at org.apache.catalina.core.StandardWrapper.loadServlet(
       at org.apache.catalina.core.StandardWrapper.load(
       at org.apache.catalina.core.StandardContext.loadOnStartup(
       at org.apache.catalina.core.StandardContext.start(
       at org.apache.catalina.core.ContainerBase.addChildInternal(
       at org.apache.catalina.core.ContainerBase.addChild(
       at org.apache.catalina.core.StandardHost.addChild(
       at org.apache.catalina.startup.HostConfig.deployWAR(
       at org.apache.catalina.startup.HostConfig.deployApps(
       at org.apache.catalina.startup.HostConfig.check(
       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
       at sun.reflect.NativeMethodAccessorImpl.invoke(
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(
       at java.lang.reflect.Method.invoke(
       at org.apache.commons.modeler.BaseModelMBean.invoke(
       at com.sun.jmx.mbeanserver.DynamicMetaDataImpl.invoke(
       at com.sun.jmx.mbeanserver.MetaDataImpl.invoke(
       at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(
       at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(
       at org.apache.catalina.manager.ManagerServlet.check(
       at org.apache.catalina.manager.ManagerServlet.deploy(
       at org.apache.catalina.manager.ManagerServlet.doPut(
       at javax.servlet.http.HttpServlet.service(
       at javax.servlet.http.HttpServlet.service(
       at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(
       at org.apache.catalina.core.ApplicationFilterChain.doFilter(
       at org.apache.catalina.core.StandardWrapperValve.invoke(
       at org.apache.catalina.core.StandardContextValve.invoke(
       at org.apache.catalina.authenticator.AuthenticatorBase.invoke(
       at org.apache.catalina.core.StandardHostValve.invoke(
       at org.apache.catalina.valves.ErrorReportValve.invoke(
       at org.apache.catalina.core.StandardEngineValve.invoke(
       at org.apache.catalina.valves.AccessLogValve.invoke(
       at org.apache.catalina.connector.CoyoteAdapter.service(
       at org.apache.coyote.http11.Http11Processor.process(
       at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(
       at org.apache.tomcat.util.threads.ThreadPool$

which makes sense since 110 (pop3) is a privileged port and you normally need to be root. I'm going to look at some UML or VServer way of doing this...

So for now I just do

zimbra@parker:~$ tomcat restart

until I work out why that works since I'm zimbra there.

from /etc/sudoers:
%zimbra ALL=NOPASSWD:/opt/zimbra/libexec/zmtomcatstart
ok then.

Now to get the code into Eclipse was easy. I created separate projects for ZimbraWebClient, ZimbraServer and Ajax

CVS Update

I did a CVS update and found that the database schema had changed. (Don't forget to do a CVS update -d for new directories)

After poking around the source tree and looking at the forums and several attempts at , I found I needed to

$ sudo cp -p ./ZimbraBuild/rpmconf/Upgrade/ /opt/zimbra/libexec/

Examining, I a saw it called perl scripts. I looked in /ZimbraBuild/rpmconf/Upgrade/upgrade.tgz, the missing one wasn't there... I found it at ./main/ZimbraServer/src/db/

$ sudo cp -p ../main/ZimbraServer/src/db/ /opt/zimbra/libexec/scripts/

Then I needed to edit /opt/zimbra/libexec/ so that the upgrade would run


if (! $newinstall && ($prevVersion ne $curVersion )) {
       progress ("Upgrading from $prevVersion to $curVersion\n");
       if (zmupgrade::upgrade($prevVersion, $curVersion)){
               progress ("UPGRADE FAILED - exiting\n");
               exit 1;
       } else {
               progress ("Upgrade complete\n");

insert the line


or use whatever build will be 1 greater than the build number for your installed version of zimbra-core. E.g. mine was 332.

$ grep zimbra-core /opt/zimbra/.install_history
1146104795: INSTALLED zimbra-core_3.1.0_GA_332.DEBIAN3.1_i386.deb


$ /opt/zimbra/libexec/

should work.

Another gotcha was that you needed to run

$ ant compress-js-files

for ZimbraWebClient to generate the zipped javascript files.

Jump to: navigation, search