Running tests
Initiating a test run
To execute a test case, use the Run, Debug or Test Builder icons on the Menubar. These represent the different available run modes.
See below for more information on these different run modes.
Alternatively, to run a group of tests, go to the Navigator view and right-click on one or more tests cases or test folders.
Monitoring the test run
Once a test run is in progress, a new tab will open in the Test Runner view to allow you to follow its progress.
In Run and Debug modes, the test run will end automatically once it has finished running its tests. You can also stop a test run early by clicking the Abort icon (
Once the test run has ended (whether passed or failed) you can generate a test run report using the Export Test Results icon (
Stop test run on error
As of Provar version 1.8.11, you can also have the test run abort automatically if any test failure is encountered. This is useful if you are running dependent test cases where if one fails there is no need to run the subsequent ones. You can enable this by clicking the Stop Test Run on Error (
Note that this is different from Error Handling on an individual test case, which allows you to skip or continue the subsequent steps of a test case if an error is encountered. Refer to error handling below for more information on this option.
Re-Running Tests
As of Provar 1.9, there is a new option in Test Runner to allow re-running a specific set of test cases based on a given test run. This means that, when test failures are encountered, they can be re-run locally without needing to set up test cycles first. This is especially useful for locally testing a change to a callable test, connection, environment or browser setting.
Once a test run has been executed, you can re-run it by navigating to the relevant test run and clicking the button Re-Run the test run.
This presents the following options:
- Rerun all: Runs all the test cases from the original test run again. The same Run Mode will be used as the original run (e.g. Debug, Test Builder).
- Rerun failures (Debug Mode): Runs only the failed test cases from the original test run again. Launches in Debug mode if the original run was in Debug or Run, or in Test Builder if it was Test Builder. In both cases the Variables view will be populated to help troubleshoot the test failures.
Note that the Rerun failures option is only available if there were failures in the original test run.
Example 1: Rerun all
Below is an example re-run where the same set of tests are run by selecting Rerun all.
In the initial test run, 3 tests were run with 1 failure.
In the repeat run, all 3 tests are run again in a different test run tab.
Example 2: Rerun failures
Below is an example re-run where only failures are run again by selecting Rerun failures.
In the initial test run, 3 tests were run with 1 failure.
In the repeat run, the single failed test is run again in a different test run tab.
Run mode
Use this mode for the official execution required for signing off a release. It can also be executed at the command line for Continuous Integration support.
Run Mode supports multiple browsers and execution will not be interfered with by any Chrome plug-ins or breakpoints. Run Mode does not capture variables, so it not recommended for debugging.
You can change the browser to run in by using the Web Browser dropdown on the Menubar.
Debug mode
Debug Mode can be used for debugging issues with a specific browser. Breakpoints are enabled and variables are captured for debugging. As with Run Mode, individual test cases or full test folders can be run, and the Web Browser can be specified from the Menubar.
Test Builder
Use Test Builder mode for building and debugging UI tests with the Test Builder. If you run a test case in Test Builder, it will execute all test steps and pause after executing the last step to allow you to add more steps. To end the test run, click the stop icon at the top of the Test Builder (
This mode should not be used for final execution sign-off, but is useful for:
- Creating UI test steps and field locations for test authoring
- Debugging using breakpoints and forwards and backwards stepping
- Capturing variables which are visible during and at the end of the test execution
- Creating Page Objects
Refer to Debugging Tests for more information on the Test Runner.
Run under ANT
Run Under ANT mode allows you to run your test cases under ANT from within Provar. It is also useful for creating your Build.xml file to run your tests under ANT on a different machine, to save creating the Build.xml file manually. Refer to Apache Ant: Generating a Build File for more information.
Headless testing in Chrome
As of Provar version 1.9.8, you can run tests headlessly in Chrome by selecting this in the Web Browser dropdown in Provar Desktop. This was previously available in Provar by configuring an Environment Variable.
Selecting this option will cause Run and Debug modes to run tests without launching the browser. Test Builder mode is unaffected by this option and will always launch a Chrome browser to enable test building.
Chrome headless in build files
The Chrome Headless option can also be selected when generating a build file using Run under ANT:
It appears in the Build.XML as follows:
Chrome headless in _chromeOptions
Previously Provar supported running tests headlessly by configuring a _chromeOptions environment variable. This option is still supported and settings defined under a _chromeOptions variable will continue to work in the same manner as before.
If a situation arises where there is a conflict between the Web Browser setting and the setting defined under _chromeOptions, preference will be given to the _chromeOptions setting.
This means that, if headless is enabled in _chromeOptions but Web Browser is set to standard Chrome (non-headless), Provar will still run Chrome in headless.
Error handling
In each test case you can specify what Provar should do in the event of an error. In the Skip on Error dropdown, you can select one of three options:
- Skip: This will stop the test Case on failure and skip to the next Test Case to be executed
- Continue: This will continue to the next Test Step. This is useful if you would like to view all the errors
- Inherit: This will have the Test Case inherit its behavior from the Calling Test (applies to Callable Tests only).
