Haymarket Media Group Company Overview

Haymarket Media Group is an international media company headquartered in London. They have over 70 market-leading brands across six countries and generate content across print, web, mobile, and live media. Haymarket is also a Salesforce customer on the Sales Cloud who uses Salesforce regression testing.

Their implementation covers common functional areas such as lead conversion, opportunity management, and account and contact creation. They use a combination of simple and complex elements such as page layouts, validation rules, Visualforce pages, and APEX. They came to Provar to improve their Salesforce regression testing process.

The Challenge of Haymarket Media Group

Haymarket found Salesforce regression testing a time-consuming process. Manual testing generally took three testers two weeks, taking 240 hours to test the entire application. This had to be repeated for every Salesforce release or release of internal functionality changes, making it a significant time investment.

Haymarket’s implementation also includes applications managed by third parties. On some occasions, these third parties would make changes to their applications without prior warning to Haymarket, which could have a negative impact on Haymarket’s processes. It could be challenging to detect issues arising from these unplanned changes. This led to a higher risk of those issues impacting the business.

Overall, Haymarket wanted to reduce the time and cost of their Salesforce regression testing and become faster in identifying and addressing issues.

The Solution in Salesforce Regression Testing

Haymarket chose Provar in the hope of speeding up their Salesforce regression testing.

Provar is a no-code test automation tool. Because Provar is tightly integrated with Salesforce, it provides a more efficient way to implement automation, intelligently mapping non-Salesforce fields, objects, and other key metadata. Minor environmental changes become easier to accommodate, reducing maintenance effort and increasing scalability.

Haymarket found Provar easy to set up and was able to build up an extensive library of test cases in a short time. The tool’s drag-and-drop interface meant that test cases could be built and updated quickly in the browser, and test cases were easy to maintain because the tool did not require any programming knowledge to use. Haymarket made roughly 150 test cases over the first three months, averaging 2 test cases a day.

Haymarket also set up daily execution reports to help them identify and address new issues quickly. Part of the team’s DevOps strategy was to execute their entire Salesforce regression testing pack automatically every night and email the results to the team for checking every morning. This was set up quickly using Provar’s ANT script generator, which can easily be set up to be run via popular Continuous Integration tools such as Jenkins. This greatly benefited the team since they could quickly check every morning to see whether any new issues had arisen since the test suite’s previous run. This made locating and resolving problems easy and increased confidence in the application’s stability.

The Results of Salesforce Regression Testing

  • 5 HOURS
  • 98 FASTER
  • 5.8 FTE SAVED

Haymarket significantly reduced its Salesforce regression testing time and effort by using Provar. Through Provar, their Salesforce regression testing time went from 240 hours to just 5: an overall reduction of 98%. There was an even bigger reduction in test effort since the entire regression pack was scheduled to run automatically and did not require human intervention. This allowed the test team to focus on more creative and exploratory testing.

Using Provar, the team could detect issues faster and with more confidence. Thanks to test scheduling and daily execution reports set up through Jenkins and ANT, the Haymarket team had access to a daily report of regression test results. This allowed them to detect new issues quickly and proactively.

Testimonial

“We are very pleased with Provar as a tool. We found it easy to set up and use, and we’ve seen a significant reduction in our Salesforce regression testing time since its adoption. Provar’s daily execution reports have been particularly effective for us, as we can see full regression test results every morning just by opening our email. Our ability to detect and respond rapidly to issues has been greatly improved.”

– Rob Burns at Haymarket

Technical Solution

Ease of Use

Provar is a no-code tool that allows you to build your tests directly in the browser. Using Provar’s point-and-click Test Builder tool, you can create your tests in the browser and make edits. This makes test case creation and debugging fast and easy.

Environment Management

Provar stores environment-specific details are stored centrally as Environment Variables and accesses them automatically when the tester chooses their environment. This means that test cases can be executed across multiple environments in just a few clicks.

Maintainability

Traditional test automation requires programming skills, and technical information such as field locators has to be hard coded. Provar is integrated into Salesforce’s metadata, meaning field locators can be automatically suggested. Changes to cosmetic elements, such as page layouts, no longer cause unexpected breakages, reducing maintenance time and effort. Test cases can also be callable by other tests, which increases the usability and stability of the test suite.

Execution Reports Straight to Your Inbox

Provar has features that allow the generation of all the configurations needed to execute tests via ANT. These integrate with the most common Continuous Integration tools, such as Jenkins. These results can then be emailed daily to help testers detect new issues quickly and confidently.

Integration Testing

Within the test scenarios, ensuring that all parameterization was tested within each PDF document generated by Salesforce was essential. These were sent as attachments in emails and also embedded in certain Visualforce pages. Provar tried these with the Gmail connector and used regular expression APIs to check the values, making this process faster and simpler.