5 Steps for Evaluating a Test Automation Tool

Conducting a proof of value (POV) for any new technology tool can be a daunting task. Mala Punyani, Salesforce DevOps Manager at Crowdstrike and a Provar champion, has done this many times in her career. In this resource, we will discuss strategies for how to evaluate a  Salesforce test automation to help you scale quality at your organization. 

First, remember that most trial licenses usually last two weeks or less. So, it’s essential to have a sharp plan to discover if a test automation tool is going to fit the needs of your organization. Mala offered five steps for reaching a well-informed decision by making the most out of this trial period.

1. Develop a POV plan

When developing a POV for a new tool, always remember the potential to be disruptive to a team’s regular workflow. Outlining a detailed plan gives your testing team a clear picture of what to expect and important milestones in the process. Many people go into a trial period not even knowing what they need in a test automation tool, be clear on what you want to achieve in this limited period of time.

2. Prioritize your needs

An automation tool needs to work for your current needs and then scale to support growth – define your success criteria as a team. Remember to consider how the tool will work for not just your current workflow and team, but scale with future teams too.

3. Determine your use cases

It’s wise to limit the number of use cases you will automate during the POV. Specifically, choose the top 3 use cases that are part of the daily flow of the primary Salesforce users in your organization.

4. Identify the stakeholders

When deciding whom from your organization should participate in the evaluation, consider who is writing the test cases and who is running the test cases. Make sure the testers you choose have adequate time in their work schedules to devote to the evaluation. Going over the plan with them will help them not only gain buy-in, but it will assure they understand the expectations and the POV timeline.

Success may look different to different stakeholders and you will want to have a feedback system in place. It’s good to include product managers for business implications, as well as members of the development team.

Understanding the skill level required for the testers evaluating the automation tool will help anticipate the learning curve and set realistic expectations. Most likely, your organization won’t want to have to hire new people or completely retrain their current team because a new automation tool has been adopted. Your current team should be able to start working with the tool immediately, without having to write a lot of new code to begin implementing. 

On the other hand, it is useful to know how easy it is to customize the test automation tool, because some of your more savvy testers are going to want to write their own modules. Customer support can help your team understand the broader capabilities of the tool beyond the use cases you’re working on during your trial. 

5. Engage with customer support

Customer support is one of the main reasons you’re choosing to invest in a test automation tool. Set up regular calls with customer support and Solution Engineers for vendor tool during your trial license period to assess their long-term use cases with the tool. Do your research to find out what other customers are saying about the services they provide.

Other Considerations

There are some other questions Mala suggests companies ask when considering a testing automation tool. One is, what other software is the tool able to integrate with besides Salesforce? It seems obvious, but if you can avoid the need for multiple automation tools in your workflow, the process will be more streamlined. Consider how the tool can extend beyond your teams to bring greater efficiency to areas of the business that you may not have initially thought about and how will the tool help your teams scale for the future ?   

Lastly, and specifically to test automation tools for Salesforce, it’s very important to ask how the tool handles changes, updates, and the annual releases of Salesforce. There are always people who are resistant to change. The best way to handle these colleagues goes back to Mala’s first piece of advice: have a plan. Make it a priority to understand what their concerns are and address those concerns in your regular communication throughout the testing period. Knowing the benefits they will gain from adoption of an automation testing tool, and showing them the results as you go, will help to build the confidence that will be required at the time you’re ready to invest in the tool. Expressing to them the cost of keeping the status quo by comparing current results to the results the tool will deliver will give these hesitant adopters a reason to get excited. 

If you’d like to watch my conversation with Mala, our interview is available on YouTube.

Ready to start a Proof of Value with Provar? Book a VIP session with our Salesforce continuous testing experts today.