Testing: Selenium: Running ZimbraSelenium: Difference between revisions

No edit summary
No edit summary
Line 1: Line 1:
{{Archive}}==General Information==
{{Archive}}==General Information==


ZimbraSelenium is the automation test harness for verifying the browser based Zimbra clients.
zm-selenium is the automation test harness for verifying the browser based Zimbra clients.


==Prerequisites==
==Prerequisites==


===Downloading and Building ZimbraSelenium===
===Downloading and Building zm-selenium===


Please follow the steps in [[Testing:_Selenium:_Building_ZimbraSelenium]].
Please follow the steps in [[Testing:_Selenium:_Building_ZimbraSelenium]].
Line 34: Line 34:
=== Tests Configuration===
=== Tests Configuration===


The file located in ZimbraSelenium/conf/conf.properties is the master configuration file. Every parameter change required to run the tests properly is configurable using this properties file.  
The file located in zm-selenium/conf/conf.properties is the master configuration file. Every parameter change required to run the tests properly is configurable using this properties file.  
Configure the test harness with the basic parameters required to run properly against your desired version of Zimbra by changing the following variables in the file:
Configure the test harness with the basic parameters required to run properly against your desired version of Zimbra by changing the following variables in the file:


* server: This is the URL in which the Zimbra install to be tested is running. Please notice that the same must be reachable from the location in which the Selenium RC server is running (localhost, [http://saucelabs.com Sauce OnDemand], a remote machine).
* server.host: This is the name of server in which the zimbra is installed. Please notice that the same must be reachable from the client machine in which test cases will be executed.
* serverMachineName: The machine in which the Selenium RC server will be running (localhost, sauceondemand or a reachable ip/domain).
* browser: The browser in which the tests should run. This is the standard Selenium browser string.
* browser: The browser in which the tests should run. This is the standard Selenium browser string.
* locale: The locale in which the installed version of Zimbra is running (en_US, ch, es_Es, etc).
* locale: The locale in which the installed version of Zimbra is running (en_US, ch, es_Es, etc).
Line 46: Line 45:
Be sure to set client resolution to 1024 X 768 or higher, otherwise the zimbra client will default to the HTML client (See also https://bugzilla.zimbra.com/show_bug.cgi?id=27521).
Be sure to set client resolution to 1024 X 768 or higher, otherwise the zimbra client will default to the HTML client (See also https://bugzilla.zimbra.com/show_bug.cgi?id=27521).


==Running ZimbraSelenium==
==Running zm-selenium==


===Running from Eclipse===
===Running from Eclipse===
The ZimbraSelenium source tree includes a build.xml file that has targets to execute the test suite.
The zm-selenium source tree includes a build.xml file that has targets to execute the test suite.


# Open ZimbraSelenium/build.xml
# Open zm-selenium/build.xml
# Open the "outline view", if not already open
# Open the "outline view", if not already open
## Window -> Show View -> Outline
## Window -> Show View -> Outline
Line 58: Line 57:
# Right-click on "Run-ExecuteHarnessMain" in the Outline, and select "Run"
# Right-click on "Run-ExecuteHarnessMain" in the Outline, and select "Run"


Here is an example build.xml "Run-ExecuteHarnessMain" configuration, which runs the "sanity" and "smoke" address book test cases for the Zimbra Mobile Client suite.
Here is an example build.xml "Run-ExecuteHarnessMain" configuration, which runs the "sanity" and "smoke" mail compose test cases for Zimbra Ajax client suite.


<pre>
<pre>
<target name="Run-ExecuteHarnessMain" depends="jar" description="Run all tests per specified arguments">
<target name="Run-ExecuteHarnessMain" depends="jar" description="Run all tests per specified arguments">
<property name="jarfile" value="${dist.lib.dir}/zimbraselenium.jar"/>
<property name="jarfile" value="${dist.lib.dir}/zimbraselenium.jar"/>
<property name="pattern" value="projects.mobile.tests.addressbook"/>
<property name="pattern" value="ajax.test.mail.compose"/>
<property name="groups" value="always,sanity,smoke"/>
<property name="groups" value="always,sanity,smoke"/>
<echo>Executing ...</echo>
<echo>Executing ...</echo>
Line 79: Line 78:
===Running from Eclipse (Dev Environment)===
===Running from Eclipse (Dev Environment)===


The ZimbraSelenium source tree is designed to run in the '''Zimbra Development Environment'''.  The Dev Environment is used to develop and build the Zimbra product.  When developing web clients or zimlets, the Zimbra Selenium test suite can be used to [http://en.wikipedia.org/wiki/Sanity_testing#Software_development Sanity Test] code changes.
The zm-selenium source tree is designed to run in the '''Zimbra Development Environment'''.  The Dev Environment is used to develop and build the Zimbra product.  When developing web clients or zimlets, the Zimbra Selenium test suite can be used to [http://en.wikipedia.org/wiki/Sanity_testing#Software_development Sanity Test] code changes.


To set up the Dev Environment, use the instructions found in the Zimbra source depot under ZimbraServer/docs/INSTALL-*.txt, for example ZimbraServer/docs/INSTALL-win.txt.  Once you have the environment up and running, follow the steps in this section to run the Zimbra Selenium suite.
To set up the Dev Environment, use the instructions found in the Zimbra source depot under ZimbraServer/docs/INSTALL-*.txt, for example ZimbraServer/docs/INSTALL-win.txt.  Once you have the environment up and running, follow the steps in this section to run the Zimbra Selenium suite.
Line 85: Line 84:
Some test suite setup needs to be completed to execute the suite.
Some test suite setup needs to be completed to execute the suite.


# Open ZimbraSelenium/conf/config.properties
# Open zm-selenium/conf/config.properties
# Modify the server settings for the client URL (i.e. <nowiki>http://localhost:7070</nowiki>)
# Modify the server settings for the client URL (i.e. <nowiki>http://localhost:7070</nowiki>)
## Set server.scheme to http or https (by default, the dev env uses http)
## Set server.scheme to http or https (by default, the dev env uses http)
Line 92: Line 91:
# Modify the admin name to use the PC name
# Modify the admin name to use the PC name
## Set adminName to the admin email address (by default, the dev env uses admin@hostname, where hostname is the computer name.  You can check this value by logging into the admin console, usually <nowiki>https://localhost:7071</nowiki> with username/password of admin/test123, then searching for the admin account, which will show the email address as something like "admin@mypcname")
## Set adminName to the admin email address (by default, the dev env uses admin@hostname, where hostname is the computer name.  You can check this value by logging into the admin console, usually <nowiki>https://localhost:7071</nowiki> with username/password of admin/test123, then searching for the admin account, which will show the email address as something like "admin@mypcname")
# Open ZimbraSelenium/build.xml
# Open zm-selenium/build.xml
# Locate the "Run-ExecuteHarnessMain (dev)" target
# Locate the "Run-ExecuteHarnessMain (dev)" target
## This target is pre-configured to run the sanity suite
## This target is pre-configured to run the sanity suite
Line 136: Line 135:


===Running from STAF===
===Running from STAF===
The ZimbraSelenium test suite can be configured using [http://staf.sourceforge.net/ STAF].
The zm-selenium test suite can be configured using [http://staf.sourceforge.net/ STAF].


====Build the ZimbraSelenium STAF jar====
====Build the zm-selenium STAF jar====


# Open ZimbraSelenium/build.xml
# Open zm-selenium/build.xml
# Open the "outline view", if not already open
# Open the "outline view", if not already open
## Window -> Show View -> Outline
## Window -> Show View -> Outline
# Locate the "staf-jar" target
# Locate the "staf-jar" target
# Right-click on "staf-jar" in the Outline, and select "Run"
# Right-click on "staf-jar" in the Outline, and select "Run"
# The jar file should be built (ZimbraSelenium/build/dist/zimbra-6.0.0/lib/zimbraselngstaf.jar)
# The jar file should be built (zm-selenium/build/dist/lib/zimbrastaf.jar)


<pre>
<pre>
Line 151: Line 150:
jar:
jar:
jar-staf-selenium:
jar-staf-selenium:
     [copy] Copying 697 files to C:\P4\main\ZimbraSelenium\build\staf\selenium\STAF-INF\classes
     [copy] Copying 697 files to C:\git\zm-selenium\build\staf\selenium\STAF-INF\classes
     [copy] Copied 171 empty directories to 1 empty directory under C:\P4\main\ZimbraSelenium\build\staf\selenium\STAF-INF\classes
     [copy] Copied 171 empty directories to 1 empty directory under C:\git\zm-selenium\build\staf\selenium\STAF-INF\classes
     [copy] Copying 23 files to C:\P4\main\ZimbraSelenium\build\staf\selenium\STAF-INF\jars
     [copy] Copying 23 files to C:\git\zm-selenium\build\staf\selenium\STAF-INF\jars
       [jar] Building jar: C:\P4\main\ZimbraSelenium\build\dist\zimbra-6.0.0\lib\zimbraselngstaf.jar
       [jar] Building jar: C:\git\zm-selenium\build\dist\lib\zimbrastaf.jar
jar-staf:
jar-staf:
BUILD SUCCESSFUL
BUILD SUCCESSFUL
Line 163: Line 162:
==== Configure the STAF jar ====
==== Configure the STAF jar ====


Configure zimbraselngstaf.jar with STAF
Configure zimbrastaf.jar with STAF


# Open C:\STAF\bin\STAF.cfg using a text editor
# Open C:\STAF\bin\STAF.cfg using a text editor
# Configure the service by adding "service seleniumm library JSTAF execute /P4/main/ZimbraSelenium/build/dist/zimbra-6.0.0/lib/zimbraselngstaf.jar"
# Configure the service by adding "service seleniumm library JSTAF execute /git/zm-selenium/build/dist/lib/zimbrastaf.jar"
# Stop STAF
# Stop STAF
## C:\STAF> staf local shutdown shutdown
## C:\STAF> staf local shutdown shutdown
Line 190: Line 189:




service selenium library JSTAF execute /P4/main/ZimbraSelenium/build/dist/zimbra-6.0.0/lib/zimbraselngstaf.jar
service selenium library JSTAF execute /git/zm-selenium/build/dist/lib/zimbrastaf.jar


</pre>
</pre>
Line 201: Line 200:
## C:\STAF> STAF local SELENIUM HELP
## C:\STAF> STAF local SELENIUM HELP
# Run a suite
# Run a suite
## C:\STAF> STAF local SELENIUM EXECUTE SERVER server.com ROOT /p4/main/ZimbraSelenium JARFILE /p4/main/ZimbraSelenium/build/dist/zimbra-6.0.0/lib/zimbraselenium.jar PATTERN projects.mobile.tests.addressbook GROUP always GROUP sanity LOG /tmp
## C:\STAF> STAF local SELENIUM EXECUTE SERVER server.com ROOT /git/zm-selenium JARFILE /git/zm-selenium/build/dist/lib/zimbraselenium.jar PATTERN ajax.tests.mail.compose GROUP always GROUP sanity LOG /tmp


Example help:
Example help:
Line 207: Line 206:
<pre>
<pre>


C:\STAF>
C:\STAF>
C:\STAF>
C:\STAF>STAF local SELENIUM HELP
C:\STAF>STAF local SELENIUM HELP
Line 236: Line 234:


C:\STAF>
C:\STAF>
C:\STAF>
C:\STAF> STAF local SELENIUM EXECUTE SERVER server.com ROOT /git/zm-selenium JARFILE  
C:\STAF> STAF local SELENIUM EXECUTE SERVER server.com ROOT /p4/main/ZimbraSelenium JARFILE  
/git/zm-selenium/build/dist/lib/zimbraselenium.jar PATTERN  
/p4/main/ZimbraSelenium/build/dist/zimbra-6.0.0/lib/zimbraselenium.jar PATTERN  
ajax.tests.mail.compose GROPU always GROUP sanity LOG /tmp
projects.mobile.tests.addressbook GROPU always GROUP sanity LOG /tmp
Response
Response
---------------
---------------
Line 284: Line 281:
===== Running the tests =====
===== Running the tests =====


# cd to ZimbraSelenium dir
# cd to am-selenium dir
# run ant targets:  
# run ant targets:  
  ant "Run tests"
Runs both ajax and html suite


   ant "Run ajax tests"  
   ant "Run ajax tests"  
Copies i18n and test data, compiles the code, creates the jar file and runs ajax suite
Copies i18n and test data, compiles the code, creates the jar file and runs ajax suite


  ant "Run html tests"
Copies i18n and test data, compiles the code, creates the jar file and runs html suite


{{Article Footer|unknown|11/7/2007}}
{{Article Footer|unknown|11/7/2007}}

Revision as of 06:54, 26 May 2017

General Information

zm-selenium is the automation test harness for verifying the browser based Zimbra clients.

Prerequisites

Downloading and Building zm-selenium

Please follow the steps in Testing:_Selenium:_Building_ZimbraSelenium.

Once the project built and no dependency errors were found, you are almost ready to run your tests. Just configure the tests to work properly in your environment as explained in the next section and you'll be ready to test.

Software

STAF

The Zimbra Selenium harness depends on STAF being installed on both the test client and zimbra server. See the design overview for more details.

Install STAF on both the client and server. And configure security level 5 for communication from the client to the server. For example, if the client IP address is 192.168.1.1 and the server IP address is 192.168.1.2, run this command to set security level 5:


root@192.168.1.1:~# ssh root@192.168.1.2
root@192.168.1.2's password: *****
root@192.168.1.2:~# staf local trust set machine 192.168.1.1 level 5
Response
--------

root@192.168.1.2:~#


Tests Configuration

The file located in zm-selenium/conf/conf.properties is the master configuration file. Every parameter change required to run the tests properly is configurable using this properties file. Configure the test harness with the basic parameters required to run properly against your desired version of Zimbra by changing the following variables in the file:

  • server.host: This is the name of server in which the zimbra is installed. Please notice that the same must be reachable from the client machine in which test cases will be executed.
  • browser: The browser in which the tests should run. This is the standard Selenium browser string.
  • locale: The locale in which the installed version of Zimbra is running (en_US, ch, es_Es, etc).

Screen Resolution

Be sure to set client resolution to 1024 X 768 or higher, otherwise the zimbra client will default to the HTML client (See also https://bugzilla.zimbra.com/show_bug.cgi?id=27521).

Running zm-selenium

Running from Eclipse

The zm-selenium source tree includes a build.xml file that has targets to execute the test suite.

  1. Open zm-selenium/build.xml
  2. Open the "outline view", if not already open
    1. Window -> Show View -> Outline
  3. Locate the "Run-ExecuteHarnessMain" target
  4. Set the jarfile, pattern, and groups desired to be executed
  5. Right-click on "Run-ExecuteHarnessMain" in the Outline, and select "Run"

Here is an example build.xml "Run-ExecuteHarnessMain" configuration, which runs the "sanity" and "smoke" mail compose test cases for Zimbra Ajax client suite.

<target name="Run-ExecuteHarnessMain" depends="jar" description="Run all tests per specified arguments">
	<property name="jarfile" value="${dist.lib.dir}/zimbraselenium.jar"/>
	<property name="pattern" value="ajax.test.mail.compose"/>
	<property name="groups" value="always,sanity,smoke"/>
	<echo>Executing ...</echo>
	<java classname="framework.core.ExecuteHarnessMain" classpathref="class.path" fork="true" failonerror="true">
		<arg line="-j '${jarfile}' -p ${pattern} -g ${groups} -l conf/log4j.properties"/>				
	</java>
</target>

Notes:

  • "always" must be included in the "groups" property
  • "zimbraselenium.jar" must be specified in as the "jarfile" property

Running from Eclipse (Dev Environment)

The zm-selenium source tree is designed to run in the Zimbra Development Environment. The Dev Environment is used to develop and build the Zimbra product. When developing web clients or zimlets, the Zimbra Selenium test suite can be used to Sanity Test code changes.

To set up the Dev Environment, use the instructions found in the Zimbra source depot under ZimbraServer/docs/INSTALL-*.txt, for example ZimbraServer/docs/INSTALL-win.txt. Once you have the environment up and running, follow the steps in this section to run the Zimbra Selenium suite.

Some test suite setup needs to be completed to execute the suite.

  1. Open zm-selenium/conf/config.properties
  2. Modify the server settings for the client URL (i.e. http://localhost:7070)
    1. Set server.scheme to http or https (by default, the dev env uses http)
    2. Set server.host to localhost (by default, the dev env uses localhost)
    3. Set server.port to the web server port number (by default, the dev env uses 7070)
  3. Modify the admin name to use the PC name
    1. Set adminName to the admin email address (by default, the dev env uses admin@hostname, where hostname is the computer name. You can check this value by logging into the admin console, usually https://localhost:7071 with username/password of admin/test123, then searching for the admin account, which will show the email address as something like "admin@mypcname")
  4. Open zm-selenium/build.xml
  5. Locate the "Run-ExecuteHarnessMain (dev)" target
    1. This target is pre-configured to run the sanity suite
  6. Right-click on "Run-ExecuteHarnessMain (dev)" in the Outline, and select "Run"


Here is an example of the config.properties changes:

...

# Define the Zimbra server
server.scheme=http
server.host=localhost
server.port=7070


ZimbraLogRoot=test-output
adminName=admin@mypcname
adminPwd=test123
isAppliance=false

...


After selecting "Run", the results will be displayed in the Eclipse console output. Here is an example:

     ...
     [java] 100500 [main] INFO  org.openqa.jetty.util.Container  - Stopped org.openqa.jetty.jetty.servlet.ServletHandler@10ffb38
     [java] 100547 [main] INFO  org.openqa.jetty.util.Container  - Stopped HttpContext[/wd,/wd]
     [java] 100547 [main] INFO  org.openqa.jetty.util.Container  - Stopped org.openqa.jetty.jetty.Server@1e328e0
     [java] 101500 [main] WARN  framework.core.SeleniumService  - Selenium server is stopped
     [java] 101500 [main] INFO  framework.core.ExecuteHarnessMain  - Total Tests:   7
     [java] Total Passed:  35
     [java] Total Failed:  0
     [java] Total Skipped: 0
     [java] Duration: 215 seconds
BUILD SUCCESSFUL
Total time: 4 minutes 14 seconds


Running from STAF

The zm-selenium test suite can be configured using STAF.

Build the zm-selenium STAF jar

  1. Open zm-selenium/build.xml
  2. Open the "outline view", if not already open
    1. Window -> Show View -> Outline
  3. Locate the "staf-jar" target
  4. Right-click on "staf-jar" in the Outline, and select "Run"
  5. The jar file should be built (zm-selenium/build/dist/lib/zimbrastaf.jar)
...
jar:
jar-staf-selenium:
     [copy] Copying 697 files to C:\git\zm-selenium\build\staf\selenium\STAF-INF\classes
     [copy] Copied 171 empty directories to 1 empty directory under C:\git\zm-selenium\build\staf\selenium\STAF-INF\classes
     [copy] Copying 23 files to C:\git\zm-selenium\build\staf\selenium\STAF-INF\jars
      [jar] Building jar: C:\git\zm-selenium\build\dist\lib\zimbrastaf.jar
jar-staf:
BUILD SUCCESSFUL
Total time: 33 seconds


Configure the STAF jar

Configure zimbrastaf.jar with STAF

  1. Open C:\STAF\bin\STAF.cfg using a text editor
  2. Configure the service by adding "service seleniumm library JSTAF execute /git/zm-selenium/build/dist/lib/zimbrastaf.jar"
  3. Stop STAF
    1. C:\STAF> staf local shutdown shutdown
  4. Start STAF
    1. C:\STAF> stafproc

Staf.cfg example:

# Turn on tracing of internal errors and deprecated options
trace enable tracepoints "error deprecated"

# Enable TCP/IP connections
interface ssl library STAFTCP option Secure=Yes option Port=6550


# Set default local trust
trust level 2 default
trust machine local://local level 5

# Add default service loader
serviceloader library STAFDSLS


service selenium library JSTAF execute /git/zm-selenium/build/dist/lib/zimbrastaf.jar

Execute the tests

Use STAF commands to run tests.

  1. Get the service help
    1. C:\STAF> STAF local SELENIUM HELP
  2. Run a suite
    1. C:\STAF> STAF local SELENIUM EXECUTE SERVER server.com ROOT /git/zm-selenium JARFILE /git/zm-selenium/build/dist/lib/zimbraselenium.jar PATTERN ajax.tests.mail.compose GROUP always GROUP sanity LOG /tmp

Example help:


C:\STAF>
C:\STAF>STAF local SELENIUM HELP

Response
--------
StafTest Service Help

EXECUTE SERVER <servername|IP address> ROOT <ZimbraSelenium path> JARFILE <path> 
PATTERN <projects.zcs.tests> [ GROUP <always|sanity|smoke|full> ]* 
[ LOG <folder> ] [ LOG4J <properties file> ]

QUERY -- TBD: should return statistics on active jobs

HALT <TBD> -- TBD: should stop any executing tests

HELP



C:\STAF>


Example execute:

 

C:\STAF>
C:\STAF> STAF local SELENIUM EXECUTE SERVER server.com ROOT /git/zm-selenium JARFILE 
/git/zm-selenium/build/dist/lib/zimbraselenium.jar PATTERN 
ajax.tests.mail.compose GROPU always GROUP sanity LOG /tmp
Response
---------------
Total Tests: 2
Total Passed: 2
Total Failed: 0
Total Skipped: 0



C:\Documents and Settings\zimbra>

C:\STAF>

Running from the command line

Running with ant commands

Installing Ant

You need Ant 1.7+ to be installed on your system.

Notice: If you are running Mac OS X 10.5 or newer you can safely skip this step — the system comes with Ant 1.7.0 pre-installed.

If you are running an older version on Mac OS X you still need to install Ant 1.7: the pre-installed Ant version is too much behind. Follow the instructions described in the "Other Systems" section.

Ubuntu

If you are running Ubuntu 8.04 "Hardy Heron", the easier way to install Ant 1.7 is to run:

 sudo apt-get install ant ant-optional

Other Systems

Install Ant 1.7.x and make sure it is in your PATH:

Download the binary distribution.
Unpack the binary distribution in the directory of your choice.
Add <where you unpacked the zip file>/apache-ant-1.7.0/bin to your PATH environment variable.
Check that your Ant installation is OK:

 ant -version
 Apache Ant version 1.7.0 compiled on December 13 2006
Running the tests
  1. cd to am-selenium dir
  2. run ant targets:
 ant "Run ajax tests" 

Copies i18n and test data, compiles the code, creates the jar file and runs ajax suite


Verified Against: unknown Date Created: 11/7/2007
Article ID: https://wiki.zimbra.com/index.php?title=Testing:_Selenium:_Running_ZimbraSelenium Date Modified: 2017-05-26



Try Zimbra

Try Zimbra Collaboration with a 60-day free trial.
Get it now »

Want to get involved?

You can contribute in the Community, Wiki, Code, or development of Zimlets.
Find out more. »

Looking for a Video?

Visit our YouTube channel to get the latest webinars, technology news, product overviews, and so much more.
Go to the YouTube channel »

Jump to: navigation, search