Parallel execution in Microsoft Azure Pipelines using Test Plans

Microsoft Azure Pipelines is a cloud service that you can use to automatically build and test your code project and make it available to other users. It works with just about any language or project type. This support article describes how to execute parallel execution in Microsoft Azure Pipeline Test Plans.  

Azure Pipelines combines continuous integration (CI) and continuous delivery (CD) to constantly and consistently test and build your code and ship it to any target.

You can build multiple Test Plans and run a repeatable collection of tests per each release cycle and make global changes to the environment settings such as browser settings, build number and build server. Plus, you can get consolidated reports of your results.

Once you have the build.xml file exported using a Test Plan, you can parameterize the Test Plan in the build.xml file. In the following example, we have two Test Plans i.e., Smoke and Regression. These Test Plans contain the test case instances that you can run as part of your smoke and regression testing. We will provide the Test Plan to be executed as part of the pipeline.

Prerequisites in Microsoft Azure Pipelines using Test Plans

Step 1: Configure the build.xml file to parameterize the Test Plan as suggested below in the screenshot.

Step 2: Configure Azure Pipelines with an ANT task. To get started, click Variables and create a variable TestPlan with the value of: 

Regression,Smoke

Note: The value is the name of the new Test Plan that you created. It could be different for you. 

Above: View of the Variables tab within Azure Pipelines. 

Step 3: Click Tasks and then the selected agent job. 

Step 4: Under Parallelism, select Multi-Configuration.

Step 5: Complete the following configuration.

Multipliers: TestPlan

Note: This is the variable name being defined during Step 2.

Maximum number of agents: Provide the number of agents that you want to execute test cases in parallel.

Step 6: Select the ANT task.

Step 7: Use the following information to complete the configuration.

• Ant build file: <Path to build file>\build.xml
• Options: -DTestPlan=$(TestPlan). This will replace the TestPlan variable mentioned in the build.xml file.
• Check Publish to Azure Pipelines under JUnit Test Results
• Test results files: <Path to Result file from build.xml>\junit.xml

Step 8: If you execute Azure Pipelines, you should now see that two agents are running – each executing their respective Test Plan from the build.xml file.