Provar is uniquely able to run across multiple environments used in a typical software development lifecycle. In order to align with Salesforce’s source-driven development approach using scratch orgs, and traditional org-driven development, we’ve delivered ProvarDX™ to make this integration seamless for our users.
ProvarDX™ can be installed as a Visual Studio Code Extension or installed using our NPM package. Installation of the Visual Studio Code extension will automatically install the required NPM package when the first command is used.
Please follow the instructions on those two listings for the latest installation instructions and especially read carefully and follow the prerequisites before installation. If you have issues installing and getting the Salesforce CLI working, please contact Salesforce for assistance first before proceeding with ProvarDX™.
We also provide ProvarDX™ as an open-source solution which you are free to modify for non-commercial use. Please note that any local customizations you make may prevent you from benefiting from the upgrades we make and cannot be supported by our service teams if you have issues. We welcome community contributions to improve the feature.
After installing ProvarDX™, the easiest way to check if the plugin is installed correctly is to open a command line window and type the following:
sfdx plugins --core
Within the listed output you should see an entry for @provartesting/provardx.
This will also tell you what version of the ProvarDX™ library you are using. Note these version numbers are independent of the Provar Desktop and CLI versions. In general:
- ProvarDX™ version (0.2.2 above) affects the set of commands and flags you can use, but not the operation. i.e. Allows you to specify which tests are run and what options you want to pass the Provar TestRunner.
- Provar Desktop/CLI version (e.g. v2.1.1) affects the behavior of the commands when you invoke them, i.e. what TestRunner does when it receives the command to start.
A simple way to verify if the plugin is working correctly is using the following command:
This will check if the default property file is well formed and helps to verify that the installation is working. Note: You will get an error/warning if you have not yet created a provardx-properites.json file.
Once ProvarDX™ is installed, think about the commands you want to use day-to-day. Our Salesforce DX Integration article is on hand to guide you through one such scenario or feel free to reach out to our Success Team for advice.
Updates and patching
Once the ProvarDX™ npm package is installed into your Salesforce CLI, ProvarDX™ will be updated automatically to the latest published version whenever you perform a sfdx update command (you can choose to exclude plugins during update too). You must ensure that the compatible version of Provar is also installed locally and that your Provar Home is set correctly for successful operation.
Updates to both our ProvarDX™ npm package and Visual Studio Code extension may be published outside of changes to the core Provar application distribution.
Creating your DX property file
ProvarDX™ property files provide context information about the environments you wish to execute test cases in. You should consider creating more than one property file to represent different stages of your DevOps pipeline, or for executing different test packs. If no property file is specified then ProvarDX™ will automatically look for a file called provardx-properties.json.
The permitted structure and enumerations for ProvarDX™ property files have been defined using a JSON schema file. Updates may be made over time by us to this schema file and you should consider using the ProvarDX™ validate command to check your property files’ compatibility whenever you upgrade ProvarDX™.
You can also override some of the property attributes at the command line if you wish. This allows you to maintain flexibility and avoid the need to edit your property file to change the tests you wish to execute or the environment you wish to test against.
The easiest way to create a ProvarDX™ property file is to use the Navigator Export as ProvarDX context menu option on the selected folders or test cases to include:
We recommend that you always use relative paths to define the location of your cache, Java page objects and test case files. This ensures your property file can be used locally and synchronized with your CI/CD platform for both Mac and Windows users.
The following known issues should be noted
- LoginAs Salesforce connection overrides are not yet supported
- Passwords need to be supplied (with secrets encryption) for non-scratch org connections
- ProvarDX™ currently requires a local installation of Provar Desktop. We plan to ensure it is bundled in our Provar CLI distributions as soon as possible to reduce the need to copy over .jar files onto server environments.
- ProvarDX™ is an unsigned plugin, you will get a warning on installation, as you do for every Salesforce CLI plugin today. Salesforce does not yet support plugin signing by third-party publishers for the Salesforce CLI but we expect this to come in the near future.
-o | --connectionoverridesparameter is inconsistent between metadatacache and runtests commands, we’re reworking this to match the simpler solution defined for metadatacache and hope to have an update soon.
Loglevelattribute returns the same level of detail regardless of the setting. This is awaiting forthcoming changes to Provar Test Runner to enhance logging and control of log messages.
- General information
- Licensing Provar
- Provar trial guide and extensions
- Using Provar
- API testing
- Behavior-driven development
- Creating and importing projects
- Creating test cases
- Custom table mapping
- Debugging tests
- Defining a namespace prefix on a connection
- Defining proxy settings
- Environment management
- Exporting test cases into a PDF
- Exporting test projects
- Managing test steps
- Namespace org testing
- Provar desktop
- Provar Test Builder
- Refresh and Recompile
- Reload Org Cache
- Running tests
- Searching Provar with find usages
- Secrets management and encryption
- Setup and teardown test cases
- Tags and Service Level Agreements (SLAs)
- Test cycles
- Test plans
- Testing browser options
- Tooltip testing
- Using the Test Palette
- Test Palette introduction
- Control test steps
- List compare
- Read test step
- String test steps
- UI Test Steps
- Using custom APIs
- Callable tests
- Data-driven testing
- Page objects
- Block locator strategies
- Introduction to XPaths
- Creating an XPath
- Label locator strategies
- Maintaining page objects
- Mapping non-Salesforce fields
- Page object operations
- Refresh and reselect field locators in Test Builder
- Using Java method annotations for custom objects
- Applications testing
- Database testing
- Document testing
- Email testing
- Mobile testing
- OrchestraCMS Testing
- Salesforce CPQ testing
- ServiceMax testing
- Skuid testing
- Vlocity testing
- Webservices testing
- Introduction to test scheduling
- Apache Ant
- Configuration for sending emails via the Provar Command Line Interface (CLI)
- Continuous integration
- Azure DevOps
- Running a Provar CI task in Azure DevOps
- Configuring the Provar secrets password in Microsoft Azure Pipelines
- Parallel execution in Microsoft Azure Pipelines using multiple build.xml files
- Parallel execution in Microsoft Azure Pipelines using targets
- Parallel execution in Microsoft Azure Pipelines using Test Plans
- Bitbucket Pipelines
- Circle CI
- GitHub Actions
- Running a Provar CI task in GitHub Actions
- Remote Trigger in GitHub Actions
- Parameterization using Environment Variables in GitHub Actions
- Parallel Execution in GitHub Actions using multiple build.xml files
- Parallel Execution in GitHub Actions using Targets
- Parallel Execution in GitHub Actions using Test Plan
- Parallel Execution in GitHub Actions using Job Matrix
- GitLab CI
- Travis CI
- Parallel Execution
- Running Provar on Linux
- Salesforce DX
- Team foundation server
- Version control
- Zephyr Cloud and Server
- Salesforce testing
- Adding a Salesforce connection
- Assert Page Error Messages on Add/Edit Product
- Dynamic Forms
- Internationalization support
- List and table testing
- Salesforce Release Updates
- Salesforce Lightning Testing
- Salesforce Lightning Web Component (LWC) locator support
- Salesforce console testing
- Visualforce Testing
- Testing best practices
- Configurations and permissions
- Error messages
- Licensing, installation and firewalls
- Test Builder and test cases
- Release notes