Integrating with TestRail
To efficiently manage your test cases and runs, you can execute test cases and record test results using TestRail. So, you can update the execution status of the test cases in TestRail from Provar, when a test case gets Pass/Fail/Skipped during the execution. We integrate Provar with TestRail using TestRail REST endpoints and this is done by using the web services connection available in Provar.
In brief, the TestRail integration allows you to link a test case in Provar to a test case in TestRail. After execution, the test run results are shown in TestRail and give you the full and thorough clarity of your test results.
Briefly, the overall workflow of TestRail integration with Provar is as given below:
1) Create test cases in TestRail.
2) Create a TestRail connection in Provar.
3) Create a TestRun set up.
4) Execute the test cases in Provar and update the test case results in TestRail.
Above: Depicting overall steps and workflow for TestRail integration.
To begin with, we have first created three test cases in TestRail.
- Create Account with ID as C1.
- Create Opp with ID as C2.
- Create Test with ID as C3.
These C1, C2 and C3 represent test case ID in TestRail.
Note: Users must pass the exact test case ID generated by TestRail (after a test case is created in TestRail) in Provar to get the status updated after the execution from Provar.
Above: Displaying test cases created in TestRail.
Steps for integrating TestRail with Provar
Step 1: Add a TestRail connection.
To add a TestRail connection, select Generic Web Service from the Connection Type drop-down menu.
Note: Go to Test Settings. On the Connections tab, click ‘+’ to add a new connection. For more information, please see Web Service connection.
You will see the Add new Connection window as displayed below. To add a TestRail connection, select Generic Web Service from the Connection Type drop-down menu.
Above: Adding a TestRail connection in Provar.
Step 2: Add an authentication.
In the URL field, enter the TestRail URL. Select Authentication Type from the drop-down menu and enter the relevant authentication details.
Note: TestRail supports two authentication types:
- Username and Password
- Username and API Key
In this example, we have selected Basic Authentication with username and password. For more information, please see authentication types.
Above: Adding an Authentication Type.
Step 3: Test the connection.
Click Test Connection to check if the connection is successful to ensure the communication between Provar and TestRail is created properly.
Above: View of the Test Connection dialog box.
Step 4: Use the TestRail connection in your test case.
To use the TestRail connection in your test case, drag and drop Web Connect and Web Request (REST) API’s in your test case from the Test Palette. For more information, please see web services.
Above: Using Web Connect and Web Request (REST) API’s in the test case.
Step 5: Select the TestRail connection.
When you drag and drop the Web Connect API in your test case from the Test Palette, a Test Step parameters screen is displayed on the right hand side.
In the Connection field, select the TestRail web services connection from the drop-down.
Above: Passing the TestRail connection using the Web Connect API.
Step 6: Configure the Web Request (REST) step and set up the test case.
Now, perform a REST request. Drag and drop the Web Request (REST) API in your test case from the Test Palette.
Above: Set up the Test case and perform the Web (REST) request.
In the Connection Name list, click the TestRail connection; that you passed in the above Web connect step.
Create a Test Run in TestRail through Provar in the Resource URL field with the end point: “/index.php?/api/v2/add_run/:project_id“ where :project_id is the id of the project created in TestRail. For more info, please refer to Projects.
In the example given above, we have used the Resource URL (“/index.php?/api/v2/add_run/1“ ) where 1 is the id of your project in TestRail.
- In the REST Type field, select Generic REST Service.
- In the Request Header field, enter the optional parameter for HTTP headers, as Content-Type:application/json.
- In the Method field, select Post(create).
- In the Body Style field, select JSON.
Note: The Body parameters used are:
⦁ Name: Name of the test run (Provar run (NOW)) that is created in TestRail.
⦁ Case_ids: There is an ID associated with each test case in the TestRail. We pass these ID(s) in Provar and when the test case execution is completed; the results of these test case(s) are updated in TestRail. Here, these ID(s) are 1, 2, and 3. Please see TestRail Case and Status for more information.
Also, please refer to Runs for more information on Test Runs.
Step 7: Update test case results on TestRail.
We will now update the test execution results in TestRail of the test cases (Create Account, Create Opp, Create Test) that we created in Provar. Here, we are taking an example of the Create Account test case.
Above: Updating test case results in TestRail.
To update the results in TestRail, first Get TestRail ID.
Above: Updating test case results based on the value in TestRail.
The TestRail Execution Status is defined in the Value Name field and updates the status of the test case in TestRail based on the numeric value in the Value field. i.e. (with the variable based on the value). The Status ‘1’ signifies ‘Pass’ and ‘5’ signifies ‘Fail’.
So, based on this value, the test case status is updated in TestRail. Please see Test status in TestRail for the list of all TestRail status for more information.
Above: Updating the test execution result in TestRail.
When the Test case execution is successful in Provar, the test result is updated in the TestRail or if the test case execution fails then the test result is updated in the TestRail with an error message.
Note: In the Test Step parameters screen, the status of the test case is updated in the Resource URL field with the TestRail ID “/index.php?/api/v2/add_result_for_case/(RestResponse_AddRun.id)/TestRail_id”,as displayed in the screenshot given above.
In the Body section, note the ‘status_id’ is populated with ‘TestRail_Execution_Status’ and an error message is passed, if the test case fails and an error occurs.
Step 8: View of the test results as updated and displayed in TestRail.
The view of the test case results are updated and displayed in TestRail as shown below with the screenshots.
Above: Test results as updated and displayed in TestRail.
Above: A graphical representation of the test results updated in TestRail.
- General Information
- About Provar
- Browser and driver recommendations
- Granting org permissions to Provar
- Installing Provar Automation
- Provar Success Portal
- Recent support article improvements
- Reintroduction of CLI license Check
- System requirements
- Updating Provar Automation
- Using hash functions to verify the security of Provar files
- Licensing Provar
- Provar trial guide and extensions
- Using Provar
- API testing
- Behavior-driven development
- Creating and importing projects
- Creating test cases
- Custom table mapping
- Functions
- Debugging tests
- Defining a namespace prefix on a connection
- Defining proxy settings
- Environment management
- Exporting test cases into a PDF
- Exporting test projects
- Override auto-retry for Test Step
- Managing test steps
- Namespace org testing
- NitroX
- Provar desktop
- Provar Test Builder
- Refresh and Recompile
- Reload Org Cache
- Reporting
- Running tests
- Searching Provar with find usages
- Secrets management and encryption
- Setup and teardown test cases
- Tags and Service Level Agreements (SLAs)
- Test cycles
- Test plans
- Testing browser options
- Tooltip testing
- Using the Test Palette
- Using custom APIs
- Callable tests
- Data-driven testing
- Page objects
- Block locator strategies
- Introduction to XPaths
- Creating an XPath
- JavaScript locator support
- Label locator strategies
- Maintaining page objects
- Mapping non-Salesforce fields
- Page object operations
- ProvarX™
- Refresh and reselect field locators in Test Builder
- Using Java method annotations for custom objects
- Applications testing
- DevOps
- Introduction to test scheduling
- Apache Ant
- Configuration for Sending Emails via the Provar Command Line Interface
- Continuous integration
- AutoRABIT Salesforce DevOps in Provar Test
- Azure DevOps
- Running a Provar CI Task in Azure DevOps Pipelines
- Configuring the Provar secrets password in Microsoft Azure Pipelines
- Parallel Execution in Microsoft Azure Pipelines Using Multiple build.xml Files
- Parallel Execution in Microsoft Azure Pipelines using Targets
- Parallel execution in Microsoft Azure Pipelines using Test Plans
- Bitbucket Pipelines
- CircleCI
- Copado
- Docker
- Flosum
- Gearset DevOps CI/CD
- GitHub Actions
- Integrating GitHub Actions CI to Run Provar CI Task
- Remote Trigger in GitHub Actions
- Parameterization using Environment Variables in GitHub Actions
- Parallel Execution in GitHub Actions using Multiple build.xml Files
- Parallel Execution in GitHub Actions using Targets
- Parallel Execution in GitHub Actions using Test Plan
- Parallel Execution in GitHub Actions using Job Matrix
- GitLab Continuous Integration
- Travis CI
- Jenkins
- Execution Environment Security Configuration
- Provar Jenkins Plugin
- Parallel Execution
- Running Provar on Linux
- Reporting
- Salesforce DX
- Git
- Team foundation server
- Version control
- Salesforce testing
- Adding a Salesforce connection
- Assert Page Error Messages on Add/Edit Product
- Dynamic Forms
- Internationalization support
- List and table testing
- Salesforce Release Updates
- Salesforce Lightning Testing
- Salesforce Lightning Web Component (LWC) locator support
- Salesforce console testing
- Visualforce Testing
- Connection Best Practices
- Performance Best Practices
- Testing Best Practices
- Troubleshooting
- Browsers
- Configurations and permissions
- Connections
- DevOps
- Error messages
- Administrator has blocked access to client
- JavascriptException: Javascript error
- macOS Big Sur Upgrade
- Resolving failed to create ChromeDriver error
- Resolving Jenkins license missing error
- Resolving metadata timeout errors
- Test execution fails – Firefox not installed
- Update to Opportunity field validation behaviour
- Licensing, installation and firewalls
- Memory
- Test Builder and test cases
- Provar AArch64 Mac Installer
- Release notes