Testing: SOAP: Running ZimbraQA

 - This is archive documentation, which means it is not supported or valid for recent versions of Zimbra Collaboration.

General Information

ZimbraQA is the automation test harness for verifying the Zimbra server functionality.

Downloading and Building ZimbraSelenium

Please follow the steps in Testing:_SOAP:_Building_ZimbraQA.

Tests Configuration

The test harness must be configured to communicate with your Zimbra server deployment.

The file located in ZimbraQA/conf/global.properties is the master configuration file. Every parameter change required to run the tests properly should be present and configurable on it.

Most configuration settings can be left as the defaults, however there are several settings that are likely to be different for your deployment:

• zimbraServer.name: The Zimbra server name (i.e. server.zimbra.com)
• adminServer.name: The Zimbra server name that hosts the admin account. In multinode deployments, the adminServer.name may be different than the zimbraServer.name.
• admin.mode, admin.port, admin.path: The URL mode, port, and path to access the admin SOAP interface.
• soapservice.mode, soapservice.port, soapservice.path: The URL mode, port, and path to access the client SOAP interface
• admin.user, admin.password: The global admin/password values.
• defaultdomain.name: the default domain name, i.e. zmprov gacf zimbraDefaultDomainName

A typical global.properties file will look like:

zimbraServer.name=qa49.lab.zimbra.com
adminServer.name=qa49.lab.zimbra.com
admin.mode=https
admin.port=7071
admin.path=service/admin/soap/
soapservice.mode=http
soapservice.port=80
soapservice.path=service/soap/
restservlet.path=service/home/
contentservlet.path=/service/home/~/
uploadservlet.path=service/upload?fmt=raw
csvservlet.path=/service/user/
admin.uri=https://qa49.lab.zimbra.com:7071/service/admin/soap/
mailclient.uri=http://qa49.lab.zimbra.com:80/service/soap/
restservlet.url=http://qa49.lab.zimbra.com:80/home/
contentservlet.url=http://qa49.lab.zimbra.com:80/service/content/
uploadservlet.url=http://qa49.lab.zimbra.com:80/service/upload?fmt=raw
csvservlet.url=http://qa49.lab.zimbra.com:80/service/csv
csservice.mode=http
csservice.port=80
csservice.path=outlookWebservice/Service.asmx
defaultpassword.value=test123
defaultdomain.name=qa49.lab.zimbra.com
...

Running ZimbraQA

Running a single script from build.xml

eclipse

1. Open the eclipse project
2. Open ZimbraQA/build.xml
3. Look for the target "Run SoapTestCore"
   1. Modify the "testRoot" property to point to a soap test case from ZimbraQA/data/soapvalidator/...
4. Run the "Run SoapTestCore" by right clicking on the target in the build.xml outline
5. Log files will appear in ZimbraQA/soapvalidator/...
6. Output should appear in the Console Output such as:

...
     [java] ***************************************************************************************
     [java] EXECUTING TEST: C:\home\public\P4\main\ZimbraQA\data\soapvalidator\MailClient\Auth\auth_basic.xml
     [java] REQUEST/RESPONSE Results: 11(PASS) - 0(Fail)
     [java] Tests Executed:11
     [java] Pass:11
     [java] Fail:0
     [java] Test Cases Executed:5
     [java] Pass:5
     [java] Fail:0
     [java] Script Errors:0
     [java] Test finished: PASS
BUILD SUCCESSFUL
Total time: 16 seconds

Running the full suite from build.xml

Follow the instructions for running a single file, but set "testRoot" to "data/soapvalidator"

Notes

Smoke Suite

Certain test cases are marked with "type="smoke"", which means they are the high-priority test cases. You can run the full suite and limit the test cases to only smoke by using the "-t smoke" command line argument.

Typical smoke results are (for 6.0.8):

Executed:2667
Pass:2655
Fail:12
Script Errors:0

STAF integration

Zimbra QA test harness frequently communicate with the Zimbra server using STAF. For instance, when the test harness sends a message from one account to another account, the harness uses STAF to check the MTA message queue to make sure that all messages are delivered before proceeding. Otherwise, the test harness may run faster than the Zimbra server and give false-failure results when a message is not delivered in time.

1. Install STAF on the Zimbra server.
2. Install STAF on the harness client.
3. Give full communication permissions between the two STAF instances (see the STAF documentation)