This page describes a solution for setting up integration from Gearset to execute a Jenkins CI Server Job to execute a suite of Provar tests. Optionally you can write the test results back to a custom object in Salesforce, and we’ve listed a free AppExchange package that can be used for that purpose if you wish, or you can integrate your test results into a RDBMS connection or via other web services. The process is broadly the same for any other DevOps CI/CD tool that doesn’t run ANT natively but does allow you to configure Webhook or HTTP Callouts.
If you’re using a locally hosted Jenkins instance please ensure it is within your corporate DMZ and can accept incoming connections from Salesforce. If this is the case, you can skip to the next section. The range of Salesforce IP addresses is long and ever-changing so we recommend a cloud hosted instance unless you need to execute end-to-end tests with internal systems.
An AWS instance should be configured by following the AWS Set Up a Jenkins Server guide. Note the full public AWS DNS JENKINS_URL must be used not just the IP address,
After completing your setup ensure you can access your new Jenkins Admin screen remotely using the JENKINS_URL from your local desktop browser before continuing and not just on the AWS instance using localhost:8080 or localhost:8443. If this fails you need to check your AWS Configure Security Group and ensure it’s been applied to your AWS instance. Do not proceed until this is working.
If installing onto a Windows Server you will also need to also create an Inbound Port Forwarding rule on Windows Firewall for port 8080 or 8443. Do not restrict source IP access unless you plan to whitelist every Salesforce IP address.
It is your responsibility to lock down this AWS instance and Jenkins to meet your corporate security standards. The instance must be accessible from Gearset.
Follow the Provar Setting Up Continuous Integration guide.
After configuring the server the CSRF Protection needs to be disabled in the Manage Jenkins -> Configure Global Security. The default settings for the Access Control should be left as below until you have your integration working, and then can be locked down using Matrix Based Security.
Note: By enabling Read Only Anonymous access you can allow non-authenticated users to inspect the results of the build action. Disable this if you do not want to allow this to be publicly visible to anyone with the Jenkins server URL, and set up any additional non-admin user access you require instead.
We recommend not using your Jenkins Admin user credentials for triggering remote builds, and creating a new user specifically for this purpose, using Manage Jenkins -> Manage Users to add a new user.
For the Jenkins user you want to use to trigger tests remotely, make a note of the Username and API Token to be used. The password is notrequired for API access and should never be shared externally.
Note: You need to login as the user to be used and click the Show API token BEFORE you restrict access if using the Matrix based security.
Although you can integrate with the Jenkins Admin user, we strongly recommend that you create a new user identity in the Manage Jenkins -> Manage Users and limit their execution to execute build jobs only once you have your integration working and have captured the API Token as above for the new user:
Deploy your test cases to your Jenkins CI Server, or integrate with your Version Control repository within the Build job you wish to trigger.
Jenkins remote API configuration
Depending on your AWS Instance you will need to follow the standard install for either Windows or Linus to setup and license Provar for your execution environment.
You should also read carefully and follow all relevant steps in the Provar DevOps guide.
When configuring your Provar Build Job ensure the job is made available as a Remote Build and make a note of your Build Authentication Token and the Build URL. For remote invocation we recommend creating an additional user for this purpose, see
section. For simplicity we recommend using the buildWithParameters version of the Build URL to allow you to pass options for which Provar ANT target and tests to be executed.
You can also add the following optional parameters if you wish to override the Test Case Path and/or the ANT target to be executed. You’ll need to ensure you account for these in your corresponding Provar build.xml file.
You can name these parameters whatever you wish, just ensure you apply them consistently across Jenkins, the build.xml and your webhook callout from Gearset.
If you are using the optional parameters in Process Builder to pass the Test Case Path and/or ANT target make the following additional changes to your build.xml:
- For Test Path, replace
- For creating alternate ANT targets, copy the entire … block and paste underneath, updating the target name to be unique and change the test execution parameters as desired.
Within Gearset there is an option to configure an outgoing webhooks as part of a CI Job. For full documentation see the Gearset website: https://gearset.com/blog/outgoing-webhooks-for-ci
For the CI Job you wish to trigger a Provar test run, click on the link to Edit Job and then navigate to the Outgoing webhooks tab:
Complete the following:
- Outgoing webhook url: This is the Build URL from your Jenkins configuration
- Authentication: Check the Use Basic Auth and provide the Jenkins Username and API Token for the Gearset user (if you created one as advised)
Content Type and Payload are not required if you have used the buildWithParameters Jenkins URL.
NOTE! If you are passing parameters to the Jenkins Provar job then you will need to include them in your Build URL, for example:
We recommend first testing your integration without the additional parameters and then adding them once your Jenkins job is being triggered by Gearset.
We recommend you test each part of the process before running end to end. Ensure each of the following works to diagnose any issues you have:
- The server where Jenkins is deployed can be reached remotely, or at least the Jenkins instance accessed. If this is failing your AWS Security Rules or Server Firewall may be blocking connections. Ensure the Jenkins port (e.g. 8080) is permitted for Inbound Connections and re-test.
- The Provar job on the Jenkins server can be executed within the Jenkins admin console. If this is failing to examine the errors, fix the root causes and re-test.
- The Provar job on the Jenkins server can be executed remotely from the command line, using curl. If this is failing to examine the error returned. The most likely cause is the Jenkins Global Security. Try opening up access and re-testing to diagnose the issue.
- A Gearset CI Job triggers a new Jenkins build for the Provar job.
- Add the optional parameters to pass which test cases you wish to execute and/or which ANT target to invoke.
Hopefully, you now have a fully working end-to-end process to execute your Gearset based Continuous Delivery Cycle, well done!
- General information
- Licensing Provar
- Provar trial guide and extensions
- Using Provar
- API testing
- Behavior-driven development
- Creating and importing projects
- Creating test cases
- Debugging tests
- Defining a namespace prefix on a connection
- Defining proxy settings
- Environment management
- Exporting test cases
- Exporting test projects
- Managing test steps
- Namespace org testing
- Provar desktop
- Provar Test Builder
- Refresh and Recompile
- Reload Org Cache
- Running tests
- Searching Provar with find usages
- Secrets management and encryption
- Setup and teardown test cases
- Tags and Service Level Agreements (SLAs)
- Test cycles
- Test plans
- Testing browser options
- Tooltip testing
- Using the Test Palette
- Test Palette introduction
- Control test steps
- List compare
- Read test step
- String test steps
- UI Test Steps
- Using custom APIs
- Callable tests
- Data-driven testing
- Page objects
- Applications testing
- Database testing
- Document testing
- Email testing
- Mobile testing
- OrchestraCMS Testing
- Salesforce CPQ testing
- ServiceMax testing
- Skuid testing
- Vlocity testing
- Webservices testing
- Introduction to test scheduling
- Apache Ant
- Configuration for sending emails via the Provar Command Line Interface (CLI)
- Continuous integration
- Azure DevOps
- Running a Provar CI task in Azure DevOps
- Configuring the Provar secrets password in Microsoft Azure Pipelines
- Parallel execution in Microsoft Azure Pipelines using multiple build.xml files
- Parallel execution in Microsoft Azure Pipelines using targets
- Parallel execution in Microsoft Azure Pipelines using Test Plans
- Bitbucket Pipelines
- Circle CI
- GitLab CI
- Travis CI
- Parallel Execution
- Running Provar on Linux
- Salesforce DX
- Team foundation server
- Version control
- Zephyr Cloud and Server (Beta)
- Salesforce testing
- Adding a Salesforce connection
- Assert Page Error Messages on Add/Edit Product
- Internationalization support
- List and table testing
- Salesforce critical updates
- Salesforce Lightning Testing
- Salesforce Lightning Web Component (LWC) locator support
- Service Cloud console testing
- Visualforce Testing
- Testing best practices
- Configurations and permissions
- Error messages
- Licensing, installation and firewalls
- Test Builder and test cases
- Release notes