Testing: Selenium: ZimbraSelenium Overview
The Zimbra Selenium Harness is a system testing test harness for the Zimbra Web Clients. The Selenium harness uses Java, Selenium, and TestNG technologies to drive web client usage against the Zimbra server.
These Zimbra clients are tested by the Selenium Harness:
- Ajax Client
- HTML Client
- Mobile Client
- Admin Console
Javadoc can be generated from the source code tree. After building the software, run the "javadocs" target from ZimbraSelenium/build.xml. A javadoc tree will be created in ZimbraSelenium/build/generated/javadocs. Open index.html to view the javadocs.
The following folders are located in the ZimbraSelenium java project.
- conf: contains configuration files for how the harness executes
- data: contains test data files, such as MIME files, ICS files.
- jars: contains third party external jar files
- src/bin: contains test scripts, such as perl scripts and shell scripts
- src/java: contains java source code
- build.xml: project ant build file
The following major packages are located in the ZimbraSelenium java project.
- framework: basic java classes that apply to all testable clients
- projects: java classes specific to the tested clients
- staf: java classes that apply to integration with [STAF]
The framework package contains basic classes that apply to all testable clients.
"framework.core" contains the main execution method and Zimbra selenium classes.
"framework.items" contains classes that define basic Zimbra objects, such as Mail, Appointments, Contacts, Tasks, Tags, and Folders.
"framework.ui" contains abstract classes that define basic GUI objects, such as an Client Application, Application Page, Displayed Object, Form Object, Folder Tree Object.
"framework.util" contains utilities classes, such as injecting a MIME or creating a Zimbra Account.
Each web application is contained in a package in projects.
- projects.admin: java classes specific to the Admin Console
- projects.ajax: java classes specific to the Ajax client
- projects.html: java classes specific to the HTML client
- projects.mobile: java classes specific to the Mobile client
- projects.desktop: java classes specific to the Desktop client
Each project contains 3 sub-packages:
- projects.name.tests: all TestNG test classes are placed in "tests". Classes are further separated by functional areas within the package.
- projects.name.ui: all abstract classes from "framework.ui" are placed in "ui".
- projects.name.core: all non-test classes and non-ui classes are placed in the core package
The Selenium Harness is designed with a Business Logic layer that abstracts the GUI implementation from the Test Case method. With the abstraction layer, test case maintenance is reduced when the GUI implementation is changed. For example, to compose a new mail, the test method code may look like:
//// This is an example of a test method to compose a new mail // Create the application object Application app = new ZimbraAjaxClientApplication(); // Click the new button, get the compose page ComposePage compose = app.mailPage.toolbar(Button.NEW); // Fill out the compose page compose.to = "email@example.com"; compose.subject = "itinerary"; compose.body = "Meet me at the Zimbra Offices at 10:00 AM"; // Send the message compose.send();
The test case methods use general end-user steps - the 'business logic' that does not change in an email application. The test case methods are not required to use any reference of the GUI implementation or selenium data (such as locators). The Application, ZimbraAjaxClientApplication, and ComposePage classes are defined in the test harness, and those classes contain the logic to drive the specified user actions.
For most basic test cases, it is suggested to use the test harness building blocks within the test methods. However, a test method may be verifying a feature that is not defined in the test harness, and in those cases, the test methods should invoke lower level Selenium requests.
Test Method Structure
Test methods normally use 3 steps:
- Data setup
- GUI action
- GUI or Data verification
In Data setup, test case preconditions are set up. When a mailbox requires data to be created, it is preferred to use SOAP, REST, LMTP, etc. to set up such data which is much faster and more stable than using the GUI. Setup actions may include injecting a message into the mailbox (LMTP), sending a appointment invitation from a separate account (SOAP), posting a file to the briefcase (REST), setting the account preferences (SOAP), etc.
In GUI action, the test case actions are executed. Actions may include navigation to a separate app such as Contacts, refreshing the calendar by clicking on the Refresh button, sending a new message, etc.
In GUI or Data verification, the test verification points are measured. The Zimbra Selenium Harness uses a ZAssert class to verify. Where possible, it is preferred to use SOAP, REST, etc. to verify data (but, obviously these interfaces will not work when verifying a GUI requirement).
For example, a test case objective may be to "Verify an incoming message is placed in the inbox list."
//// ** Data setup // Inject the message using LMTP LMTPUtil.inject(testaccount, "C:\data\mimesample.txt"); String subject = "mimesample"; //// ** GUI action // Refresh the inbox list app.MailPage.toolbarClick(Button.GetMail); //// ** GUI or Data verification // Verify the message list in the GUI contains the new message List<Message> messages app.MailPage.getMessageList(); ZAssert.assertContains(messages, new Message(subject), "Verify the new message appears in the inbox list");
Ajax Application Pages
For reference, these screen captures are listed in this wiki page. The harness javadoc pages reference these images.
General Ajax Page
The following diagram shows the general ajax page. Also included are outlines for:
- "MainPage" (Red)
- "MailFolderTree" (Orange)
- "MiniCal" (Green)
- "MailPage" (Blue).
The following diagram shows the general "MailPage" view. The "MailPage" includes elements within the red box.
The following diagram shows the breakdown of the terms:
- Toolbar (red)
- Columns (green)
- List (blue)
The following diagram shows the "DisplayMail" object (red) after selecting (LEFT_CLICK) on list item (green).
The following diagram shows the "ComposeMail" object including the compose (red) and toolbar (green).
The following diagram shows an example of clicking on item (green) and selecting from the context menu (red).
// Example code to reply to a message using the context menu app.MailPage.listClick(subject, Button.RIGHT_CLICK, Button.Reply);
Zimlets Page - Social