Running Provar tests on Jenkins
There are multiple options available for executing your Provar tests on the Jenkins server.
Option 1: Provar ANT libraries on Jenkins server
You should also read carefully and follow all relevant steps in the Introduction to Test Scheduling.
Setting up continuous integration for your Provar project is always recommended. This will help you schedule test runs, get consistent reporting and allow you to execute test cases on a remote server.
There are a few additional steps to keep in mind for use with your RM tool; this is because these tools will be sending build parameters to customize how the Provar tests are run, which tests are run, etc. You can add any additional build parameters as necessary for your use case.
First, create your new job in Jenkins using the type Freestyle Job and add a few build parameters. In order to add build parameters, you must first check the box This project is parameterized.
These are all recommended build parameters as they will be passed from Salesforce to Jenkins to execute the Provar test cases.
TEST_PLAN should be the name of the test plan that you wish to execute in the Provar test project. You can also use TEST_PATH and specify this as a variable/parameter inside of your build.xml. This part is up to the user, and depends on if you are using Provar Test Plans or not.
ANT_TARGET is the name of the ANT target inside of the build.xml file that is used to execute Provar tests via ANT. This is defaulted to runtests and will likely be unchanged unless you have defined multiple targets. The most common use case for this is parallel/distributed execution.
DEPLOYMENT_ID is the RM tool’s deployment ID that is created to execute the URL callout. The deployment phase will need to be built out externally using a RM tool in your Salesforce org. This can be left as the default for now.
BUILD_FILE will be the relative path name of the build file you wish to execute via ANT. This is defaulted to ANT/build.xml, which is the name of the build.xml file Provar generates in the desktop application. You can change this to match your build file.
Now that all of the build parameters have been added, a proper Source Code Management (SCM) can be configured in this Jenkins job. We will be using Git SCM for this guide.
Adjust your settings to match your Git repository that was setup inside of Provar (referenced in the Application Layer of the Architecture section of your RM Tool’s guide).
If you are planning on triggering this Jenkins job from another tool, such as Copado, Flosum, or Gearset, then you will need to add the Trigger builds remotely build trigger.
The next step will be to add an authentication token that will be used to authenticate when calling the Jenkins job. This is only for triggering builds remotely from another tool.
This needs to be set to the NAME of the API token that you generated previously.
Once you set this token, make sure you save the URL that is displayed below the field and the token itself to a local file (using Notepad or any similar software). This URL will be used in a later section of this guide.
You can read more about different build triggers here.
The last mandatory section will be to add the build action itself.
In this section (and in any field inside of the Jenkins job), the build parameters can be utilized using the following syntax: $PARAMETER_NAME. This is the step to execute your Provar tests using the parameters passed from Copado.
If you would like to store the results of test runs and utilize the JUnit plugin in Jenkins (which you should install as a recommended plugin), then you’ll need to add one more step in the Post-build Actions.
In the Post-build Actions section at the bottom of the build, add the step called Publish JUnit test results report.
For the Test report XMLs field, put the following value to find all Provar test results:
Typically you would want to check the box to allow for empty test results, that way the build wouldn’t show as failed if no tests were run. The rest of the settings you can configure however you wish for your use case.
Save the job and test the job by triggering it manually first. If all goes well, you should see a very nice Test Result view as shown below.
You can drill down to the test results even more by clicking the Latest Test Result link.
For a more detailed guide on the Jenkins set up and job configuration, see this support article.
Option 2: Provar on Docker
If you wish to execute your test cases inside of a Docker container, please refer to our Docker guide. It is best to utilize this option for any new Jenkins server setups. This will save both time and maintenance on this server since the Dockerfile is managed by Provar. The last part of the Docker guide has detailed instructions on how to implement this solution in conjunction with Jenkins.
- General information
- Licensing Provar
- Provar trial guide and extensions
- Using Provar
- API testing
- Behavior-driven development
- Creating and importing projects
- Creating test cases
- Custom table mapping
- Debugging tests
- Defining a namespace prefix on a connection
- Defining proxy settings
- Environment management
- Exporting test cases into a PDF
- Exporting test projects
- Managing test steps
- Namespace org testing
- Provar desktop
- Provar Test Builder
- Refresh and Recompile
- Reload Org Cache
- Running tests
- Searching Provar with find usages
- Secrets management and encryption
- Setup and teardown test cases
- Tags and Service Level Agreements (SLAs)
- Test cycles
- Test plans
- Testing browser options
- Tooltip testing
- Using the Test Palette
- Test Palette introduction
- Control test steps
- List compare
- Read test step
- String test steps
- UI Test Steps
- Using custom APIs
- Callable tests
- Data-driven testing
- Page objects
- Block locator strategies
- Introduction to XPaths
- Creating an XPath
- Label locator strategies
- Maintaining page objects
- Mapping non-Salesforce fields
- Page object operations
- Refresh and reselect field locators in Test Builder
- Using Java method annotations for custom objects
- Applications testing
- Database testing
- Document testing
- Email testing
- Mobile testing
- OrchestraCMS Testing
- Salesforce CPQ testing
- ServiceMax testing
- Skuid testing
- Vlocity testing
- Webservices testing
- Introduction to test scheduling
- Apache Ant
- Configuration for sending emails via the Provar Command Line Interface (CLI)
- Continuous integration
- Azure DevOps
- Running a Provar CI task in Azure DevOps
- Configuring the Provar secrets password in Microsoft Azure Pipelines
- Parallel execution in Microsoft Azure Pipelines using multiple build.xml files
- Parallel execution in Microsoft Azure Pipelines using targets
- Parallel execution in Microsoft Azure Pipelines using Test Plans
- Bitbucket Pipelines
- GitHub Actions
- Running a Provar CI task in GitHub Actions
- Remote Trigger in GitHub Actions
- Parameterization using Environment Variables in GitHub Actions
- Parallel Execution in GitHub Actions using multiple build.xml files
- Parallel Execution in GitHub Actions using Targets
- Parallel Execution in GitHub Actions using Test Plan
- Parallel Execution in GitHub Actions using Job Matrix
- GitLab CI
- Travis CI
- Execution Environment Security Configuration
- Parallel Execution
- Running Provar on Linux
- Salesforce DX
- Team foundation server
- Version control
- Zephyr Cloud and Server
- Salesforce testing
- Adding a Salesforce connection
- Assert Page Error Messages on Add/Edit Product
- Dynamic Forms
- Internationalization support
- List and table testing
- Salesforce Release Updates
- Salesforce Lightning Testing
- Salesforce Lightning Web Component (LWC) locator support
- Salesforce console testing
- Visualforce Testing
- Performance Best Practices
- Testing best practices
- Configurations and permissions
- Error messages
- Administrator has blocked access to client
- macOS Big Sur upgrade issue
- Resolving failed to create ChromeDriver error
- Resolving Jenkins license missing error
- Resolving metadata timeout errors
- Test execution fails – Firefox not installed
- Update to Opportunity field validation behaviour
- Licensing, installation and firewalls
- Test Builder and test cases
- Release notes
- Version 2.7.0 Summer ’22
- Version 2.6.3
- Version 2.6.0 Spring ’22
- Version 2.5.0 Winter ’22
- Version 2.4.1
- Version 2.4.0 Summer ’21
- Version 2.3.1
- Version 2.3.0 Spring ’21
- Version 2.2.1
- Version 2.2.0 Winter ’21
- Version 2.1.1
- Version 2.1.0 Summer ’20
- Version 2.0.5
- Version 2.0.4
- Version 2.0.3 Spring ’20
- Version 1.9.12
- Version 1.9.11