Difference between revisions of "Testing: Selenium: Running ZimbraSelenium"

(Execute the tests)
Line 213: Line 213:
StafTest Service Help
StafTest Service Help
EXECUTE SERVER <servername|IP address> ROOT <ZimbraSelenium path> JARFILE <path>  
EXECUTE SERVER <servername|IP address> ROOT <zm-selenium path> JARFILE <path>  
PATTERN <projects.zcs.tests> [ GROUP <always|sanity|smoke|full> ]*  
PATTERN <projects.zcs.tests> [ GROUP <always|sanity|smoke|full> ]*  
[ LOG <folder> ] [ LOG4J <properties file> ]
[ LOG <folder> ] [ LOG4J <properties file> ]

Revision as of 06:56, 26 May 2017

General Information

zm-selenium is the automation test harness for verifying the browser based Zimbra clients.


Downloading and Building zm-selenium

Please follow the steps in Testing:_Selenium:_Building_ZimbraSelenium.

Once the project built and no dependency errors were found, you are almost ready to run your tests. Just configure the tests to work properly in your environment as explained in the next section and you'll be ready to test.



The Zimbra Selenium harness depends on STAF being installed on both the test client and zimbra server. See the design overview for more details.

Install STAF on both the client and server. And configure security level 5 for communication from the client to the server. For example, if the client IP address is and the server IP address is, run this command to set security level 5:

root@ ssh root@
root@'s password: *****
root@ staf local trust set machine level 5


Tests Configuration

The file located in zm-selenium/conf/conf.properties is the master configuration file. Every parameter change required to run the tests properly is configurable using this properties file. Configure the test harness with the basic parameters required to run properly against your desired version of Zimbra by changing the following variables in the file:

  • server.host: This is the name of server in which the zimbra is installed. Please notice that the same must be reachable from the client machine in which test cases will be executed.
  • browser: The browser in which the tests should run. This is the standard Selenium browser string.
  • locale: The locale in which the installed version of Zimbra is running (en_US, ch, es_Es, etc).

Screen Resolution

Be sure to set client resolution to 1024 X 768 or higher, otherwise the zimbra client will default to the HTML client (See also https://bugzilla.zimbra.com/show_bug.cgi?id=27521).

Running zm-selenium

Running from Eclipse

The zm-selenium source tree includes a build.xml file that has targets to execute the test suite.

  1. Open zm-selenium/build.xml
  2. Open the "outline view", if not already open
    1. Window -> Show View -> Outline
  3. Locate the "Run-ExecuteHarnessMain" target
  4. Set the jarfile, pattern, and groups desired to be executed
  5. Right-click on "Run-ExecuteHarnessMain" in the Outline, and select "Run"

Here is an example build.xml "Run-ExecuteHarnessMain" configuration, which runs the "sanity" and "smoke" mail compose test cases for Zimbra Ajax client suite.

<target name="Run-ExecuteHarnessMain" depends="jar" description="Run all tests per specified arguments">
	<property name="jarfile" value="${dist.lib.dir}/zimbraselenium.jar"/>
	<property name="pattern" value="ajax.test.mail.compose"/>
	<property name="groups" value="always,sanity,smoke"/>
	<echo>Executing ...</echo>
	<java classname="framework.core.ExecuteHarnessMain" classpathref="class.path" fork="true" failonerror="true">
		<arg line="-j '${jarfile}' -p ${pattern} -g ${groups} -l conf/log4j.properties"/>				


  • "always" must be included in the "groups" property
  • "zimbraselenium.jar" must be specified in as the "jarfile" property

Running from Eclipse (Dev Environment)

The zm-selenium source tree is designed to run in the Zimbra Development Environment. The Dev Environment is used to develop and build the Zimbra product. When developing web clients or zimlets, the Zimbra Selenium test suite can be used to Sanity Test code changes.

To set up the Dev Environment, use the instructions found in the Zimbra source depot under ZimbraServer/docs/INSTALL-*.txt, for example ZimbraServer/docs/INSTALL-win.txt. Once you have the environment up and running, follow the steps in this section to run the Zimbra Selenium suite.

Some test suite setup needs to be completed to execute the suite.

  1. Open zm-selenium/conf/config.properties
  2. Modify the server settings for the client URL (i.e. http://localhost:7070)
    1. Set server.scheme to http or https (by default, the dev env uses http)
    2. Set server.host to localhost (by default, the dev env uses localhost)
    3. Set server.port to the web server port number (by default, the dev env uses 7070)
  3. Modify the admin name to use the PC name
    1. Set adminName to the admin email address (by default, the dev env uses admin@hostname, where hostname is the computer name. You can check this value by logging into the admin console, usually https://localhost:7071 with username/password of admin/test123, then searching for the admin account, which will show the email address as something like "admin@mypcname")
  4. Open zm-selenium/build.xml
  5. Locate the "Run-ExecuteHarnessMain (dev)" target
    1. This target is pre-configured to run the sanity suite
  6. Right-click on "Run-ExecuteHarnessMain (dev)" in the Outline, and select "Run"

Here is an example of the config.properties changes:


# Define the Zimbra server



After selecting "Run", the results will be displayed in the Eclipse console output. Here is an example:

     [java] 100500 [main] INFO  org.openqa.jetty.util.Container  - Stopped org.openqa.jetty.jetty.servlet.ServletHandler@10ffb38
     [java] 100547 [main] INFO  org.openqa.jetty.util.Container  - Stopped HttpContext[/wd,/wd]
     [java] 100547 [main] INFO  org.openqa.jetty.util.Container  - Stopped org.openqa.jetty.jetty.Server@1e328e0
     [java] 101500 [main] WARN  framework.core.SeleniumService  - Selenium server is stopped
     [java] 101500 [main] INFO  framework.core.ExecuteHarnessMain  - Total Tests:   7
     [java] Total Passed:  35
     [java] Total Failed:  0
     [java] Total Skipped: 0
     [java] Duration: 215 seconds
Total time: 4 minutes 14 seconds

Running from STAF

The zm-selenium test suite can be configured using STAF.

Build the zm-selenium STAF jar

  1. Open zm-selenium/build.xml
  2. Open the "outline view", if not already open
    1. Window -> Show View -> Outline
  3. Locate the "staf-jar" target
  4. Right-click on "staf-jar" in the Outline, and select "Run"
  5. The jar file should be built (zm-selenium/build/dist/lib/zimbrastaf.jar)
     [copy] Copying 697 files to C:\git\zm-selenium\build\staf\selenium\STAF-INF\classes
     [copy] Copied 171 empty directories to 1 empty directory under C:\git\zm-selenium\build\staf\selenium\STAF-INF\classes
     [copy] Copying 23 files to C:\git\zm-selenium\build\staf\selenium\STAF-INF\jars
      [jar] Building jar: C:\git\zm-selenium\build\dist\lib\zimbrastaf.jar
Total time: 33 seconds

Configure the STAF jar

Configure zimbrastaf.jar with STAF

  1. Open C:\STAF\bin\STAF.cfg using a text editor
  2. Configure the service by adding "service seleniumm library JSTAF execute /git/zm-selenium/build/dist/lib/zimbrastaf.jar"
  3. Stop STAF
    1. C:\STAF> staf local shutdown shutdown
  4. Start STAF
    1. C:\STAF> stafproc

Staf.cfg example:

# Turn on tracing of internal errors and deprecated options
trace enable tracepoints "error deprecated"

# Enable TCP/IP connections
interface ssl library STAFTCP option Secure=Yes option Port=6550

# Set default local trust
trust level 2 default
trust machine local://local level 5

# Add default service loader
serviceloader library STAFDSLS

service selenium library JSTAF execute /git/zm-selenium/build/dist/lib/zimbrastaf.jar

Execute the tests

Use STAF commands to run tests.

  1. Get the service help
  2. Run a suite
    1. C:\STAF> STAF local SELENIUM EXECUTE SERVER server.com ROOT /git/zm-selenium JARFILE /git/zm-selenium/build/dist/lib/zimbraselenium.jar PATTERN ajax.tests.mail.compose GROUP always GROUP sanity LOG /tmp

Example help:


StafTest Service Help

EXECUTE SERVER <servername|IP address> ROOT <zm-selenium path> JARFILE <path> 
PATTERN <projects.zcs.tests> [ GROUP <always|sanity|smoke|full> ]* 
[ LOG <folder> ] [ LOG4J <properties file> ]

QUERY -- TBD: should return statistics on active jobs

HALT <TBD> -- TBD: should stop any executing tests



Example execute:


C:\STAF> STAF local SELENIUM EXECUTE SERVER server.com ROOT /git/zm-selenium JARFILE 
/git/zm-selenium/build/dist/lib/zimbraselenium.jar PATTERN 
ajax.tests.mail.compose GROPU always GROUP sanity LOG /tmp
Total Tests: 2
Total Passed: 2
Total Failed: 0
Total Skipped: 0

C:\Documents and Settings\zimbra>


Running from the command line

Running with ant commands

Installing Ant

You need Ant 1.7+ to be installed on your system.

Notice: If you are running Mac OS X 10.5 or newer you can safely skip this step — the system comes with Ant 1.7.0 pre-installed.

If you are running an older version on Mac OS X you still need to install Ant 1.7: the pre-installed Ant version is too much behind. Follow the instructions described in the "Other Systems" section.


If you are running Ubuntu 8.04 "Hardy Heron", the easier way to install Ant 1.7 is to run:

 sudo apt-get install ant ant-optional

Other Systems

Install Ant 1.7.x and make sure it is in your PATH:

Download the binary distribution.
Unpack the binary distribution in the directory of your choice.
Add <where you unpacked the zip file>/apache-ant-1.7.0/bin to your PATH environment variable.
Check that your Ant installation is OK:

 ant -version
 Apache Ant version 1.7.0 compiled on December 13 2006
Running the tests
  1. cd to am-selenium dir
  2. run ant targets:
 ant "Run ajax tests" 

Copies i18n and test data, compiles the code, creates the jar file and runs ajax suite

Verified Against: unknown Date Created: 11/7/2007
Article ID: https://wiki.zimbra.com/index.php?title=Testing:_Selenium:_Running_ZimbraSelenium Date Modified: 2017-05-26

Try Zimbra

Try Zimbra Collaboration with a 60-day free trial.
Get it now »

Want to get involved?

You can contribute in the Community, Wiki, Code, or development of Zimlets.
Find out more. »

Looking for a Video?

Visit our YouTube channel to get the latest webinars, technology news, product overviews, and so much more.
Go to the YouTube channel »

Jump to: navigation, search