Testing: SOAP: Running ZimbraQA
General Information
ZimbraQA is the automation test harness for verifying the Zimbra server functionality.
Downloading and Building ZimbraSelenium
Please follow the steps in Testing:_SOAP:_Building_ZimbraQA.
Tests Configuration
The test harness must be configured to communicate with your Zimbra server deployment.
The file located in ZimbraQA/conf/global.properties is the master configuration file. Every parameter change required to run the tests properly should be present and configurable on it.
Most configuration settings can be left as the defaults, however there are several settings that are likely to be different for your deployment:
- zimbraServer.name: The Zimbra server name (i.e. server.zimbra.com)
- adminServer.name: The Zimbra server name that hosts the admin account. In multinode deployments, the adminServer.name may be different than the zimbraServer.name.
- admin.mode, admin.port, admin.path: The URL mode, port, and path to access the admin SOAP interface.
- soapservice.mode, soapservice.port, soapservice.path: The URL mode, port, and path to access the client SOAP interface
- admin.user, admin.password: The global admin/password values.
- defaultdomain.name: the default domain name, i.e. zmprov gacf zimbraDefaultDomainName
A typical global.properties file will look like:
zimbraServer.name=qa49.lab.zimbra.com adminServer.name=qa49.lab.zimbra.com admin.mode=https admin.port=7071 admin.path=service/admin/soap/ soapservice.mode=http soapservice.port=80 soapservice.path=service/soap/ restservlet.path=service/home/ contentservlet.path=/service/home/~/ uploadservlet.path=service/upload?fmt=raw csvservlet.path=/service/user/ admin.uri=https://qa49.lab.zimbra.com:7071/service/admin/soap/ mailclient.uri=http://qa49.lab.zimbra.com:80/service/soap/ restservlet.url=http://qa49.lab.zimbra.com:80/home/ contentservlet.url=http://qa49.lab.zimbra.com:80/service/content/ uploadservlet.url=http://qa49.lab.zimbra.com:80/service/upload?fmt=raw csvservlet.url=http://qa49.lab.zimbra.com:80/service/csv csservice.mode=http csservice.port=80 csservice.path=outlookWebservice/Service.asmx defaultpassword.value=test123 defaultdomain.name=qa49.lab.zimbra.com ...
Running ZimbraQA
Running a single script from build.xml
- Open the eclipse project
- Open ZimbraQA/build.xml
- Look for the target "Run SoapTestCore"
- Modify the "testRoot" property to point to a soap test case from ZimbraQA/data/soapvalidator/...
- Run the "Run SoapTestCore" by right clicking on the target in the build.xml outline
- Log files will appear in ZimbraQA/soapvalidator/...
- Output should appear in the Console Output such as:
... [java] *************************************************************************************** [java] EXECUTING TEST: C:\home\public\P4\main\ZimbraQA\data\soapvalidator\MailClient\Auth\auth_basic.xml [java] REQUEST/RESPONSE Results: 11(PASS) - 0(Fail) [java] Tests Executed:11 [java] Pass:11 [java] Fail:0 [java] Test Cases Executed:5 [java] Pass:5 [java] Fail:0 [java] Script Errors:0 [java] Test finished: PASS BUILD SUCCESSFUL Total time: 16 seconds
Running the full suite from build.xml
Follow the instructions for running a single file, but set "testRoot" to "data/soapvalidator"
Notes
Smoke Suite
Certain test cases are marked with "type="smoke"", which means they are the high-priority test cases. You can run the full suite and limit the test cases to only smoke by using the "-t smoke" command line argument.
Typical smoke results are (for 6.0.8):
Executed:2667 Pass:2655 Fail:12 Script Errors:0
STAF integration
Zimbra QA test harness frequently communicate with the Zimbra server using STAF. For instance, when the test harness sends a message from one account to another account, the harness uses STAF to check the MTA message queue to make sure that all messages are delivered before proceeding. Otherwise, the test harness may run faster than the Zimbra server and give false-failure results when a message is not delivered in time.
- Install STAF on the Zimbra server.
- Install STAF on the harness client.
- Give full communication permissions between the two STAF instances (see the STAF documentation)