* part one of a two-part series takes a deep dive into choosing the right test automation solution for your team’s needs. Part One focuses on what you should look for in the technology itself, while Part Two focuses on looking beyond the technology and evaluating the vendor’s business model. Part Two will be posted on 11/17/22.
The guide below will clarify what to look for and what questions to ask to ensure you select the right solution for your team’s needs. We will start by discussing the technology, of course.
The key considerations for a product evaluation of a Salesforce-specific testing solution fall into four categories:
| Test resilience or fragility | How well does the solution morph to the needs of the rest of the organization by using a single test that can run across numerous contexts? |
| Polymorphism and reusability | How well can the system run end-to-end tests that reach adjacent non-Salesforce systems? |
| Ease of use and learning | What skills and experience levels are required to create, maintain, and run tests |
| Testing adjacent systems | How well the can system run end-to-end tests that reach adjacent non-Salesforce systems |
Let’s break this down in a little more detail.
Test Resilience or Fragility
Salesforce automatically updates its platform multiple times yearly, which can cause automated tests to break unless the testing system was designed to operate with Salesforce. And when tests break, you’re in trouble.
Here are some questions you can ask to decide if a test automation solution has the test resilience or fragility you need:
- Can a test created for Visualforce run without modification under Aura (Classic and Lightning)?
- If Salesforce changes how a page is rendered (changes to the HTML, CSS, Javascript, etc.), will a test continue to work without modification?
- If we change a Salesforce layout, will a test created for the previous page layout work without modification?
- If Salesforce changes an API used in a test, will it continue to work without modification?
Polymorphism and Reusability
The technology that makes tests resilient also enables polymorphism, where a single test can run across numerous contexts, and reusability reduces the number of tests you need to create and maintain. Reusable polymorphic tests make for a higher-quality solution that evolves quickly and easily.
Here are some questions you can ask to gauge a test automation solution’s polymorphism and reusability:
- Can I create a single test and run the test for several different user profiles or page layouts without modification or creating additional tests?
- Will a test that I create in English run without modification and without creating duplicate tests in another language like French or Japanese?
- Can a single test run without modifying or creating additional tests across PC, Mac, and mobile users on several browsers?
- Can a test validate field visibility and accessibility based on user permissions for several profiles/permission sets without modification or creating additional tests?
- Can a test created in the development environment run without modification and without creating additional tests in the validation or production environments?
Ease of Use and Learning
Ease of use and learning is another significant factor to consider in your evaluation. Many Salesforce users are considered “citizen developers,” meaning they may not be programming languages and frameworks experts. Your entire team must use your test automation solution so you can confidently deploy your tests.
Here are some questions you can ask to see if a test automation solution champions ease of use and learning:
- Are test steps added from a Salesforce screen (“right-click to add test step”)? Can you build your test in one location without toggling back and forth between multiple platforms or screens?
- Can I add a test step and see the results before adding the next step? Can I step back to remove a prior step? Can I pause and resume a test as I run it?
- If I select a button in the Salesforce user interface and right-click to add a test step, does the test step automatically default to click? Does it default to “set” for an input field? What other default actions should we capture? Does your approach automatically detect the locators, such as elements you want to test before you test, or do I have to label these elements? When running a test script, do you see the test pass/fail in real-time, or do you need to fix and rerun it to confirm the issue is resolved?
Testing Adjacent Systems
Finally, regarding the technology itself, you must evaluate a test automation solution’s ability to test adjacent systems. Every Salesforce platform connects to other enterprise systems and custom integration points, so your testing solution must work well with other systems so that you can test your workflows end-to-end.
Here are some questions that’ll help you evaluate whether your test automation solution is ideal for testing adjacent systems:
- Can I test that a triggered email was sent? Can I test email to case flow?
- Can I verify that a Salesforce action triggered a connected action in an external system, such as creating an invoice in an ERP? Can I verify that an action on our website created the correct records/activities in Salesforce?
- Can I test that an external API has been called from a Salesforce customization?
Have more questions? Connect with us today to discuss your evaluation process and what your team’s goals are. We look forward to helping you choose the right test automation tool!
