Testing: Selenium: ZimbraSelenium CodeCoverage
- ajax client
- the admin console.
Code Coverage depends on having jscoverage installed on the server in /usr/local/bin/jscoverage, which corresponds to the coverage.tool property in ZimbraSelenium/conf/config.properties.
Don't use the apt-get install jscoverage, that doesn't seem to work correctly. Instead, get the source zip and compile. Then, copy/link /usr/local/bin/jscoverage to the built files.
To manually instrument the server, the command is (as root):
/usr/local/bin/jscoverage --no-instrument=help/ /opt/zimbra/jetty/webapps/zimbra /opt/zimbra/jetty/webapps/instrumented
The harness uses jscoverage-server mode, where jscoverage.html is not required. Therefore, the code coverage does not require the zimbra application to run inside a containing window. See JSCoverage documentation for more details.
STAF must be enabled with level 5 permissions on both the client and server for coverage to be activated. STAF is used to remotely execute instrumentation commands.
The harness config.properties file has several settings that affect coverage reporting.
coverage.enabled=false coverage.tool=/usr/local/bin/jscoverage coverage.reportsource=false coverage.query=dev=1&debug=0 coverage.maxpageload.msec=90000 coverage.instrument=true
- coverage.enabled: Set to "true" or "false" to enable or disable, respectively, coverage reporting
- coverage.tool: Set to the path of jscoverage tool on the server
- coverage.reportsource: Set to "true" or "false" to enable or disable, respectively, source code line counts in the coverage report. Enabling this option greatly increases harness memory usage, since all the source code is loaded into memory.
- coverage.query: the URL query options to instruct the application to not zip the JS files. (JScoverage requires the source code to be in JS format. The tool will not work with zip format files.)
- coverage.maxpageload.msec: The Zimbra application takes much longer to load when running in code coverage mode. This timeout setting will override the default harness page load timeout, so that the harness will not report failures when the application takes too long to load.
- coverage.instrument: Set to "true" or "false" to enable or disable, respectively, server instrumentation at the beginning and end of the tests
The configuration settings can be used to control the behavior of the code coverage mechanism. For instance, when developing new scripts with code coverage enabled, you may choose to manually instrument the code, rather than the harness instrumenting the code at run-time (which takes an additional 2-3 minutes). In this case, you would likely choose these settings
coverage.enabled=true # Enable coverage coverage.instrument=false # The harness should not instrument, since it was manually done
After executing the harness, the output will be written to ZimbraSelenium/test-output/.../coverage/jscoverage.html
There are multiple files in the coverage folder, however, jscoverage.html is the main html page and jscoverage.json is the JSON file which contains the coverage metrics. See JSCoverage documentation for more details.
TIP: The report uses quite a lot of memory and processing when loaded. The Chrome browser seems to work best when viewing the report.
Here is an example report: