Building Zimbra with Maven
This wiki is a collection of notes, tips, and tricks related to the migration to the Maven build system for the KISS/9.0 release. Readers are highly encouraged to add new tips, correct outdated information, and expand on the content.
- 1 Benefits
- 2 Maven Installation
- 3 Maven Usage
- 4 Troubleshooting
The first question an engineer asks before changing anything should always be 'what problem are we trying to solve here?'. In this case, there are many inefficiencies to the current Ant build process which can be improved by using Maven. The collaboration server is comprised of dozens of third party Java libraries, and managing those libraries is a cumbersome process.
- Cleaner dependency hierarchy - We often have to trace through Bugzilla history to determine what project/module uses a particular jar file. We have a number of jar files which are most likely unused but we have no clear way to determine which ones are safe to remove. Maven requires each project to explicitly list dependencies, and gives us a tool to understand where dependencies are inherited vs. where they are directly used in our codebase.
- Standard declarative project structure - Our current Ant projects are a mishmash of repeated copy/paste scripting with small variance for each project. Maven establishes a standard project object model (POM) which is then inherited by all sub projects. This means that we do not have to reinvent standard build functions such as compile and jar for each new project and that we will reduce the number of naming variances for example where we have 'dev-dist' in one project and 'dist-dev' in another performing a similar function.
- Easier ability to trace/debug into third party libraries - Currently we maintain a collection of the source for several of the most commonly used third party jar files for use in debugging the inner workings of those libraries; however obtaining the source for additional libraries can be cumbersome; and we often overlook updating the source when we update the jar. Maven and the m2e plugin can automatically download the sources for most jar files, making it much easier to debug across third party code.
- Easier ability to upgrade in development - We frequently need to determine if a particular bug in a third party library exists in more recent versions. Manually downloading the new jar and installing in a development environment is time consuming. With Maven the same can be accomplished by updating a single version number in a single file.
- Easier ability to automate unit tests - Our Hudson continuous integration server currently has to call the 'ant test' target in a number of individual directories, and our current configuration appears to exclude a few directories which would be desirable to cover in CI. Maven allows us to create a hierarchy so 'mvn test' is called in a single directory which then automatically
We require Maven 3; and all testing to date has been done with Maven 3.2.2 and 3.0.5 versions.
To check if you have Maven installed; execute
This should return Apache Maven 3.0.0 or a higher version. If you do not have Maven, you will need to install it.
Mac OS X
Maven was included by default in 10.8 and earlier, but is not included from 10.9 on. The easiest way to install it is via homebrew:
$brew install maven
If you do not already have homebrew installed, see the homebrew website for instructions http://brew.sh/.
If you would like to attempt to install Maven by hand here is a discussion which includes some suggestions http://stackoverflow.com/questions/8826881/maven-install-on-mac-os-x
Available in apt-get/yum package managers.
If anyone is still building the server on Windows they can please fill in these details.
Maven uses the MAVEN_OPTS environment variable to pass arguments to the underlying Java process.
It sometimes helps to set a higher max memory size than default:
It also helps to set the JAVA_HOME environment variable:
Now is a great time to upgrade to Eclipse Luna if you have not done so already.
Two plugins which are highly recommended (i.e. required):
m2e dynamic source lookup: https://marketplace.eclipse.org/content/m2e-dynamic-sources-lookup
The .project/.classpath files have been updated to use the Maven build nature for projects which have been converted. The m2e plugin will allow Eclipse to interact with Maven, and actually may be installed by default in new downloads of Eclipse Luna.
The m2e dynamic source lookup plugin which allows Eclipse to resolve/fetch the sources for Maven dependencies when debugging (most Maven dependencies publish sources, although we will likely find some do not). Without this plugin you can still debug your own source but cannot easily step into third party libraries. This plugin appears to require Eclipse Luna.
Legacy Eclipse Support
If you are running an old version of Eclipse and/or cannot install the m2e plugin, you can ask Maven to generate .project and .classpath files for you. This not not ideal or recommended, but if you are stuck on an old version this could be a viable workaround. These instructions are mutually exclusive with m2e, so use one or the other but not both.
To do this, first open the files in question so they are writable, then run the eclipse:clean and eclipse:eclipse goals
p4 open .classpath
p4 open .project
mvn eclipse:clean eclipse:eclipse
Now when you open Eclipse you'll see each dependency referenced via a M2_REPO classpath variable. If Eclipse has not configured this for you, you can add it in Preferences->Java->Build Path->Classath Variables, or through the mvn eclipse:configure-workspace goal as described here: http://www.mkyong.com/maven/how-to-configure-m2_repo-variable-in-eclipse-ide/
Usage from Ant
If you don't want to use Maven directly, just call your normal Ant targets which should still work.
One note on this is that some Ant subprojects call compile/jar in parent projects and can create a bit of a cycle compiling the various Maven projects repeatedly. To avoid this pass the -Dskip.maven=true argument on the ant command line.
ant -Dskip.maven=true compile-just-my-jar
Probably will be useful with targets like dev-sync. The normal reset-all takes care of this implicitly by calling Maven once at the start of the build and then skipping it on subsequent calls in the same ant run.
Direct Maven Usage
As mentioned in the section covering the migration from Ant, most developers can simply continue to use Ant and the build process will trigger Maven when required. However, Java developers are encouraged to use new features provided by Maven to streamline their workflow.
- Build everything which is Maven enabled: From the basedir (i.e. //depot/zimbra/main) invoke
- Run unit tests
- Eventually things like
$mvn jetty:runwill be enabled so you can quickly deploy a test version of the server.
- Jetty debugging currently works via the old jetty-ant mechanism. Instructions for using jetty-maven-plugin should be straightforward and written up soon.
- Eclipse Luna seems to have the best support for debugging into Maven dependencies. More to follow on this.
- Developers add your favorite goal/target/tip/trick here.
Jars which are not available in Maven
Several existing Zimbra dependencies are not available in Maven public repositories, for example jar files which Zimbra has patched or jar files which simply are not published in Maven by their author. These are kept in Perforce under ZimbraCommon/jars-bootstrap. During an 'ant reset-all' invocation, the 'maven-seed-local-repo' target is called to install these jars in the local Maven repository. While it should generally not be necessary; if you happen to remove your Maven local repository (e.g. by deleting ~/.m2/repository) you will also want to remove ZimbraServer/mvn.seed.properties; as this is used by Ant to determine which bootstrap jars have been installed in earlier Ant runs.
You may also run the mvn-local-jars shell script to populate the local repository with these jars.
Perforce client workspace inclusions
You may run into Maven errors building the first time if you do not have all of the standard directories in your P4 client workspace. For example you may see errors such as this:
[exec] [ERROR] The goal you specified requires a project to execute but there is no POM in this directory (/Users/devuser/zimbra/p4/main). Please verify you invoked Maven from the correct directory. -> [Help 1]
Child module /home/devuser/p4/devuser-u14-main/main/ZimbraNative does not exist
If you see these errors make sure you have the following in your P4 client specification.
One of the nice features of Maven's standard lifecycle is that unit tests are run automatically during every build. In general this is great because it prevents developers from accidentally forgetting to run unit tests and submitting bad code. However, it may be necessary to skip tests under some circumstances; for example if there are sporadic/environmental failures, or if you need to build your component even though some unrelated test is failing.
In these cases, the -DskipTests=true argument can be passed to Maven or Ant.
$mvn -DskipTests=true clean install
$ant -DskipTests=true reset-all