At Dreamforce 2019, Provar hosted a fascinating session with Mala Punyani, Test Architect at Splunk, and Nicolai Rønnow Iversen, IT Architect at Alm. Brand. In that session with Splunk and Alm Brand, those architects described their strategies for implementing Provar in their organizations and the benefits they realized. Let’s find out their Splunk and Alm. Brand story.
Note: Salesforce recorded this session, so we’ll update this blog when the video is up.
Although the two architects described different approaches within their organizations, Splunk and Alm Brand achieved incredible testing effort reductions in a few short months after Provar adoption.
What Benefits Did Splunk Realize After Adopting Provar?
Splunk achieved an overall 75% reduction in test effort after three months:
Credit: Mala Punyani, Test Architect at Splunk
Let’s dig into what made Splunk so successful with Provar.
End-to-End Testing (Splunk)
In the session, Splunk described how they automated their entire purchase order flow using Provar, including quoting with Salesforce CPQ, order generation in Netsuite, and completion of the Salesforce sales cycle, including licensing, service contracts, and entitlements.
Below is the flow automated using Provar:
Credit: Mala Punyani, Test Architect at Splunk
Strategic Test Features of Provar
Splunk highlighted the following key features of Provar as particularly valuable to their test strategy.
- Test data management:
- Data-driven facility with Read/Write via/into Excel
- APIs to quickly create test data in opportunities, accounts, and other objects
- Usage of SOQL queries to return values and assertions
- Bulk Update feature via API
- Control APIs, e.g., Loop for repetitive actions
- Table creation in Provar for dynamic value assertions
- Batch execution in Salesforce with the help of Execute Apex API
- Provar Managed Dashboard
- End to end:
- Automated scenarios where multiple systems are integrated, and the data flows from one system to other
- PDF validation
- Verification of email notification triggered through Salesforce CPQ (Steelbrick) quoting system.
- Service Contract Batch Generation
For more information, refer to the Provar product page.
What Benefits Did Alm. Brand Realize after Adopting Provar?
Alm. Brand achieved a 90% reduction in hours spent on release testing after six months. Interestingly, Provar and Selenium noted a significant decrease in user-reported errors, and the number of hotfixes in production was halved compared to when they had been using Selenium.
Credit: Nicolai Rønnow Iversen, IT Architect at Alm. Brand
Solving Selenium Challenges (Alm. Brand)
Provar is designed to address some challenges of testing the Salesforce platform and solve issues where other solutions may struggle, such as reliability and reusability. Let’s listen to Nicolai on their success story with Provar.
Nicolai Rønnow Iversen, IT Architect at Alm. Brand said:
“We came from a Selenium setup that was extremely time consuming to develop and maintain. After we went from Classic to Lightning in our Salesforce org, we lost our entire test setup. Thousands of hours lost, and we had no way of reusing any of it.”
Alm. Brand described the following benefits of moving to Provar after using Selenium:
- Faster development time of test cases
- 3-12 speedier test execution than Selenium
- More resilient to changes in UI
Credit: Nicolai Rønnow Iversen, IT Architect at Alm. Brand
Alm. Brand also noted that now they create test cases, only the functionality they want to test is done in the UI. All the rest is done on an API or SOQL, which allows them to develop test cases much faster.
Nicolai noted,
“The Provar test cases execute 3-12 times faster compared to Selenium, and as an important bonus our tests are much more resilient to changes in the UI.”
For more information on Alm Brand’s approach to test case implementation, look at the Case Study: How Alm. Brand Uses Provar With GitLab and Docker to Support a Complex Salesforce Solution.
Parallel Testing with Gitlab and Docker (Alm. Brand)
Another unique element of Alm. Brand’s test strategy is using GitLab pipelines to run several tests in parallel using multiple Docker containers.
Credit: Nicolai Rønnow Iversen, IT Architect at Alm. Brand
Alm. Brand noted that running Provar in Docker containers on GitLab pipelines enabled them to quite easily spin up nine instances to execute all their main test areas simultaneously. Alm. The brand currently has 150 test cases that manage in less than 2 hours, so they run them all each night to ensure short feedback loops to their developers and configurators.
Nicolai said,
“We develop all our test cases so they are user and Org agnostic, in this way we supply our test the Org and Salesforce user as a parameter.
“Our test cases are in GitLab, and we move them up through the branches in the same pace as we move Salesforce changes, in this way we can test the different versions of Salesforce functionality as we move it up through our Orgs.”
Provar and Continuous Integration
Jon Robinson of Provar noted,
“It’s great to see the success Alm. Brand has had with GitLab and Docker. Provar aims to support every aspect of a customer’s test strategy including their different release management providers and tools, CI solutions and services.
Thank you, Splunk and Alm. Brand!
All in all a fascinating talk from two companies, Splunk and Alm Brand, having success with Provar with very different testing strategies. Thank you to Mala Punyani, Test Architect at Splunk, and Nicolai Rønnow Iversen, IT Architect at Alm. Brand, for their insightful commentary and discussion of Splunk and Alm. Brand story.
Interested in Learn More?
If you’d like to know how Provar could help your testing, view our on-demand webinar, Provar 101: Introduction to Test Automation. Join us for this introductory webinar as we demonstrate how to achieve test automation that’s easy to set up, simple to maintain, and readily integrated into your DevOps pipeline. Click here to view it now.
“”
