Parallel execution in Microsoft Azure Pipelines using Test Plans
Microsoft Azure Pipelines is a cloud service that you can use to automatically build and test your code project and make it available to other users. It works with just about any language or project type.
Azure Pipelines combines continuous integration (CI) and continuous delivery (CD) to constantly and consistently test and build your code and ship it to any target.
This support article describes how to execute parallel execution in Azure Pipelines using Provar Test Plans.
You can build multiple Test Plans and run a repeatable collection of tests per each release cycle and make global changes to the environment settings such as browser settings, build number and build server. Plus, you can get consolidated reports of your results.
Once you have the build.xml file exported using a Test Plan, you can parameterize the Test Plan in the build.xml file. In the following example, we have two Test Plans i.e., Smoke and Regression. These Test Plans contain the test case instances that you can run as part of your smoke and regression testing. We will provide the Test Plan to be executed as part of the pipeline.
Prerequisites
Step 1: Configure the build.xml file to parameterize the Test Plan as suggested below in the screenshot.
Step 2: Configure Azure Pipelines with an ANT task. To get started, click Variables and create a variable TestPlan with the value of:
Regression,Smoke
Note: The value is the name of the new Test Plan that you created. It could be different for you.
Above: View of the Variables tab within Azure Pipelines.
Step 3: Click Tasks and then the selected agent job.
Step 4: Under Parallelism, select Multi-Configuration.
Step 5: Complete the following configuration.
Multipliers: TestPlan
Note: This is the variable name being defined during Step 2.
Maximum number of agents: Provide the number of agents that you want to execute test cases in parallel.
Step 6: Select the ANT task.
Step 7: Use the following information to complete the configuration.
- Ant build file: <Path to build file>\build.xml
- Options: -DTestPlan=$(TestPlan). This will replace the TestPlan variable mentioned in the build.xml file.
- Check Publish to Azure Pipelines under JUnit Test Results
- Test results files: <Path to Result file from build.xml>\junit.xml
Step 8: If you execute Azure Pipelines, you should now see that two agents are running – each executing their respective Test Plan from the build.xml file.
- General information
- Licensing Provar
- Provar trial guide and extensions
- Using Provar
- API testing
- Behavior-driven development
- Creating and importing projects
- Creating test cases
- Custom table mapping
- Functions
- Debugging tests
- Defining a namespace prefix on a connection
- Defining proxy settings
- Environment management
- Exporting test cases into a PDF
- Exporting test projects
- Managing test steps
- Namespace org testing
- Provar desktop
- Provar Test Builder
- Refresh and Recompile
- Reload Org Cache
- Reporting
- Running tests
- Searching Provar with find usages
- Secrets management and encryption
- Setup and teardown test cases
- Tags and Service Level Agreements (SLAs)
- Test cycles
- Test plans
- Testing browser options
- Tooltip testing
- Using the Test Palette
- Using custom APIs
- Callable tests
- Data-driven testing
- Page objects
- Block locator strategies
- Introduction to XPaths
- Creating an XPath
- JavaScript locator support
- Label locator strategies
- Maintaining page objects
- Mapping non-Salesforce fields
- Page object operations
- ProvarX™
- Refresh and reselect field locators in Test Builder
- Using Java method annotations for custom objects
- Applications testing
- DevOps
- Introduction to test scheduling
- Apache Ant
- Configuration for sending emails via the Provar Command Line Interface (CLI)
- Continuous integration
- AutoRABIT
- Azure DevOps
- Running a Provar CI task in Azure DevOps
- Configuring the Provar secrets password in Microsoft Azure Pipelines
- Parallel execution in Microsoft Azure Pipelines using multiple build.xml files
- Parallel execution in Microsoft Azure Pipelines using targets
- Parallel execution in Microsoft Azure Pipelines using Test Plans
- Bitbucket Pipelines
- Circle CI
- Copado
- Docker
- Flosum
- Gearset
- GitLab CI
- Jenkins
- Travis CI
- Parallel Execution
- Running Provar on Linux
- Reporting
- Salesforce DX
- Git
- Team foundation server
- Version control
- Zephyr Cloud and Server
- Salesforce testing
- Adding a Salesforce connection
- Assert Page Error Messages on Add/Edit Product
- Dynamic Forms
- Internationalization support
- List and table testing
- Salesforce Release Updates
- Salesforce Lightning Testing
- Salesforce Lightning Web Component (LWC) locator support
- Salesforce console testing
- Visualforce Testing
- Testing best practices
- Troubleshooting
- Release notes