Haymarket Media Group Accelerates Salesforce Regression Testing with Provar

Haymarket Media Group Company Overview

Haymarket Media Group is an international media company headquartered in London. They have over 70 market-leading brands across six countries and generate content across print, web, mobile, and live media. Haymarket is also a Salesforce customer on the Sales Cloud who uses Salesforce regression testing.

Their implementation covers common functional areas such as lead conversion, opportunity management, and account and contact creation. They use a combination of simple and complex elements such as page layouts, validation rules, Visualforce pages, and APEX. They came to Provar to improve their Salesforce regression testing process.

The Challenge of Haymarket Media Group

Haymarket found Salesforce regression testing a time-consuming process. Manual testing generally took 3 testers 2 weeks, making a total of 240 hours to test the entire application. This had to be repeated for every Salesforce release or release of internal functionality changes, making it a major time investment.

Haymarket’s implementation also includes applications managed by third parties. On some occasions, these third parties would make changes to their applications without prior warning to Haymarket, which could have a negative impact on Haymarket’s processes. It could be difficult to detect issues arising from these unplanned changes. This led to a higher risk of those issues having an impact on the business.

Overall, Haymarket wanted to reduce the time and cost of their Salesforce regression testing and become faster to identify and address issues.

The Solution in Salesforce Regression Testing

Haymarket chose Provar in the hope of speeding up their Salesforce regression testing.

Provar is a no-code test automation tool. Because Provar is tightly integrated with Salesforce, it provides a more efficient way to implement automation, intelligently mapping non-Salesforce fields, objects, and other key metadata. Minor changes to an environment become easier to accommodate, reducing maintenance effort and increasing scalability.

Haymarket found Provar easy to set up and was able to build up a large library of test cases in a short time. The tool’s drag-and-drop interface meant that test cases could be built and updated quickly in the browser, and test cases were easy to maintain because the tool did not require any programming knowledge to use. Haymarket built roughly 150 test cases over the first 3 months, averaging 2 test cases a day.

Haymarket also set up daily execution reports to help them identify and address new issues quickly. Part of the team’s DevOps strategy was to execute their entire Salesforce regression testing pack automatically every night and email the results to the team for checking every morning. This was set up quickly using Provar’s ANT script generator, which can easily be set up to be run via popular Continuous Integration tools such as Jenkins. This was a great benefit to the team since they could quickly check every morning to see whether any new issues had arisen since the test suite’s previous run. This made it easy to locate and resolve issues and it increased confidence in the stability of the application.

The Results of Salesforce Regression Testing

• 5 HOURS
• 98 FASTER
• 5.8 FTE SAVED

Haymarket was able to significantly reduce its Salesforce regression testing time and effort by using Provar. Through Provar, their Salesforce regression testing time went from 240 hours to just 5: an overall reduction of 98%. There was an even bigger reduction in test effort since the entire regression pack was scheduled to run automatically and did not require human intervention. This allowed the test team to focus on more creative and exploratory testing.

Using Provar the team was also able to detect issues faster and with more confidence. Thanks to test scheduling and daily execution reports set up through Jenkins and ANT, the Haymarket team had access to a daily report of regression test results. This allowed them to detect new issues quickly and proactively.

Testimonial

“We are very pleased with Provar as a tool. We found it easy to set up and use, and we’ve seen a significant reduction in our Salesforce regression testing time since its adoption. Provar’s daily execution reports have been particularly effective for us as we can see full regression test results every morning just by opening our email. Our ability to detect and respond rapidly to issues has been greatly improved.”

– Rob Burns at Haymarket

Technical Solution

Ease of Use

Provar is a no-code tool that allows you to build your tests directly in the browser. Using Provar’s point-and-click Test Builder tool, you can build your tests step by step in the browser and make edits as you go. This makes test case creation and debugging fast and easy.

Environment Management

Provar stores environment-specific details are stored centrally as Environment Variables and accesses them automatically when the tester chooses their environment. This means that test cases can be executed across multiple environments in just a few clicks.

Maintainability

Traditional test automation requires programming skills, and technical information such as field locators has to be hard coded. Provar instead is integrated into Salesforce’s metadata, meaning that field locators can be suggested automatically. Changes to cosmetic elements such as page layouts no longer cause unexpected breakages, meaning that maintenance time and effort are reduced. Test cases can also be made callable by other tests, which increases the usability and stability of the test suite.

Execution Reports Straight to your Inbox

Provar has features that allow the generation of all the configurations needed to execute tests via ANT. These integrate with the most common Continuous Integration tools in use, such as Jenkins. These results can then be emailed on a daily basis to help testers detect new issues quickly and confidently.

Integration Testing

Within the test scenarios, it was important to ensure that all parameterization was tested within each PDF document generated by Salesforce. These were sent as attachments in emails and also embedded in certain Visualforce pages. Provar tested these with the Gmail connector and used regular expression APIs to check the values, making this process faster and simpler.