Common Challenges in Salesforce Test Automation

Salesforce is at the heart of enterprise business processes and 59% of companies are deploying a new build daily. That is why we need to identify common Salesforce test automation challenges.

Brittle automated tests remain a challenge and are costing teams time, and money, and are ultimately undermining a core Salesforce benefit – agility. Modern leaders seeking to drive revenue for the business must first address the continuous rework spiral in their software delivery.

Enjoy a conversation about the Salesforce test automation challenges and how Provar’s Salesforce-aligned architecture and solutions can help teams get out of the rework spiral with industry leader, Andy Menzies, CRO at Provar.

Question and Answer with Andy Menzies on Common Challenges in Salesforce Test Automation

Tristan Lombard: Andy, we are so thrilled that you’re here and the topic of today’s conversation. We know that many leaders out there are having some challenges with automating test automation strategies, and we wanted to talk with a leader like yourself, and we had a few questions for you if that’s all right.

Andy Menzies: Yeah, of course…go ahead.

Tristan Lombard: So we know that Salesforce is at the heart of enterprise business processes and that 59% of companies are deploying a new build daily…as a leader in this field, why is now the time for companies to start investing in Salesforce test automation?

Andy Menzies: Well, let’s be clear about this. Testing used to be a conversation that was had purely on the left-hand side of the equation i.e., development and IT… testing is a boardroom topic…there’s a direct correlation between the quality of your applications, and the metrics you use to run your business, whether it be top line, bottom line, customer satisfaction, NPS score, retention, churn.

Whatever the metric is, there’s a direct correlation between the quality of your application and your ability to compete in your marketplace.

Tristan Lombard: In your opinion, why does Provar Salesforce-aligned architecture make our solutions unique in the market?

Andy Menzies: We’ve gone to market with four key pillars if you will. Unbreakable. What do we mean by that? Well, I’ve been involved in test automation for some time now. I’ve been involved in test automation at the cutting edge where artificial intelligence and machine learning were incorporated into the technology stack. Obviously, with good reason, with motivation to improve quality generally, but what has happened with that evolution is we’ve introduced a whole lot of new work, a whole lot of time to market disadvantage. 

If people are having to build frameworks for supporting AI and they’re building digital twins or state transition models that are blueprints, digital blueprints of their application, they’re entering into a whole heap of work that they didn’t have to do. Now, what we mean by unbreakable is this fact, undisputed.

Salesforce releases multiple releases every year. You have no choice. If you go back to the days of Salesforce’s entry into the marketplace, they talked about no software. You will use the upgrades that they release, and as a consequence, if you’ve used traditional tools to build your test automation framework, they’ll break. They are brittle. Equally, if you were to build all of your test automation framework Selenium, by consequence, every time your underlying application changes, your code breaks.

You’ll enter what we call the re-work spiral. You’ll spend more time maintaining your test automation framework than you will be building that or building it. Couple that with what I’ve just talked about in terms of new approaches using AI and machine learning, for example, where you have to build a digital blueprint, you’re then doubling the amount of work effort that you need.

So why is that important? It’s important because I’ve already insinuated and again, I think it’s undisputed that the quality is directly correlated to the metrics that you use to run the business. So why introduce a whole heap of additional workload, a whole heap of additional expense to pay attention to quality? So the fact that we’re unbreakable, the fact that we work from within our DNA if you like with Salesforce, that we support the metadata, it means every time those changes occur, Provar just keeps running. It’s unbreakable.

You also mentioned polymorphism…It’s a word we borrowed from biology…where an organism can have many different forms, and the reason why we’ve stolen that term is that you can build a test script in Provar that can take on many different forms. In other words, you don’t have to rewrite that script every time that particular journey occurs in a business process.

You can use that one script that you’ve developed in a polymorphic fashion and drag and drop it into as many different scripts or journeys as you like. That once again saves a whole heap of time and therefore money.

Tristan Lombard: And I think that’s something that you touched on previously as well when you’re talking about metrics when we’re talking about digital blueprints. We hear this in the industry as well from a testing approach. Right? You can have the best test approach and still fail to help the business achieve its goals. You touched on this a little bit. I’m wondering, what advice would you give to leaders to help better align their testing goals to drive results for the business?

Andy Menzies: Yeah. I don’t know whether you guys are familiar with the term BizDevOps. So I would expect you are. I think Gartner coined the phrase many, many years ago. It should be BizDevOps because the whole idea of DevOps was bringing development and operations together to operate more harmoniously so that you were able to increase the speed and the velocity at which you were developing and releasing technology to the market.

Well, the business has a really important role to play in that. So my advice is to get business involved in building your test organization, and your test framework as early as you possibly can, and make sure that what you’re doing is identifying with what your user’s experience is in the field, and you’re maximizing the experience in terms of quality that they have.

Salespeople come to work every day and I’m talking most specifically about CRM here rather than all of the other plethora of Salesforce apps that we test. Salespeople come to work every day to hit targets, to close deals, and if you like to feed the pipeline. And they can’t do this if they’re stuck on a screen. It’s important to consider the employees as well as your customers when it comes to testing technology. What we want to do is we want to empower our employees and give them productivity tools that help them do their job more effectively. And the only way to do that is to make sure that what you deploy works and works in a way that satisfies them to deliver on that goal of productivity.

Tristan Lombard: I think that’s well said. And just as a follow-up kind of question there, when we’re thinking about usability and maintenance and scalability, I’m just wondering, what are your top three recommendations for leaders who are considering investing in new testing tools? What would you have them consider when they’re looking at investing?

Andy Menzies: Well, I mean, certainly if we’re talking about Salesforce, which I truly hope that we are on the fact that that’s what we’re all about. Firstly, ignore every other technology that’s generic. Ignore every technology that is built specifically to test anything and everything, and focus on a technology that’s been built from the ground up to test Salesforce. I think that’s really, really important. A lot of business people aren’t necessarily going to be familiar with LWC classic lightning, but not necessarily going to be familiar with some of the frameworks within Salesforce, visual force, and so on and so forth, but these are really important considerations when it comes to testing.

Because if your test tool hasn’t been built specifically to work with Salesforce, you’re going to go through a whole heap of pain trying to get it in the first instance to be representative of the app that you’re testing. And in the second instance, as I’ve alluded to already, it’s going to be really hard work and very costly to maintain.

Tristan Lombard: Yeah. And I appreciate that. And I think that was when we’re touching back on … thinking about the value props for Provar in the first place. Intuitive, right? So whether you’re a Salesforce administrator who is spending too much manual testing, pre-release, or you’re a Salesforce developer looking for custom-coded solutions, you want to have a tool that can do both and has that flexibility, and I think that’s one of the things our many customers have been saying why they love Provar and just something to consider. I had another quick question for you…You have often stated that we need to shift from POC as a term we’re using to POV in our industry. Can you elaborate on this?

Andy Menzies: In the world in which we live, where test automation I believe has already proven its value, we probably all do not from a sales cycle perspective have to do either a POC or proof of value. Certainly not a POC, which is where my belief system originates from because proof of concept as it describes is proving the concept of test automation, and I don’t think that that needs to be proven.

I think everybody has a responsibility for the quality, and I think everybody understands the relationship between quality and outcome, in this case, positive business outcomes. So proof of value, unlike a concept, is to focus on those outcomes.

Why are we testing this? What do we believe we will realize in terms of the positive business outcome as a consequence of having a higher-quality Salesforce deployment? And I think the answer to that is, as I alluded to earlier, it’s the metrics that you use to run your business. So for me, if we’re entering into a proof of value with a prospect or indeed a customer in a different geography or a different subsidiary or department, we should be focusing on the outcomes that test investment is realizing in terms of metric rather than specifically necessarily testing the features and the functions of the tool.

Now, don’t get me wrong. The features and functions of the tool drive positive business outcomes. So for example, a feature of our technology is the fact that we work with Salesforce metadata. That means that every time there is a new release and forgive me for potentially repeating myself, but every time a new release is made available to you, which you have to take, your tests if they were built in something other than Provar will break. So the fact that we worked with the metadata means that we simply refresh that metadata, and then all of the tests that you’ve put a lot of time and effort into building, potentially, if you’re looking for 80, 90% coverage of your app, that’s still a reasonable investment. That’ll work with Provar.

Tristan Lombard: So follow-up question to that, Andy, is whether there was a recent world quality report stating that automation is the biggest bottleneck holding back the evolution of QA and testing today. And we see this sometimes when it comes to effective implementation for any leader. And I’m wondering from your experience, what’s the number one mistake that you see leaders make when it comes to implementing their test automation solutions and how does Provar help address this issue?

Andy Menzies: I don’t want to spend all of my time trying to necessarily differentiate between Provar and every other technology on the market, because there are clearly some advantages in some other technologies, particularly outside of Salesforce. But I think as test automation has evolved, one thing that’s remained common outside of Provar is that it actually takes a whole heap of time to build the environment, to build the framework, and to get going. As a consequence, what a lot of folks are doing is simply building a happy path. They’re building the regression pack and that’s it. They’re not necessarily shining the torch into all of the journeys that the users are taking as they interact or navigate their way around the application.

So from my perspective, the number one Provar advantage for business leaders is time to market advantage. I mean, let’s just put this in perspective and there’s a whole heap of references that we can give the audience in terms of Splunk’s use of our technology, ALM brand’s use of our technology, and many, many others. And I would encourage everybody to visit our website and have a look at those case studies.

I’ve been hiring a lot of people into Provar of late…and within two days, these guys are productive in using our product. Within two days, these guys are building test suites, using Provar, and running those test suites in order to determine the quality of the application that’s deployed to the field. That is a distinct advantage that I don’t believe any other vendor can claim to the same extent that we can. So business leaders, leaders, and anyone that cares about quality, particularly in and around Salesforce applications need to think about how quickly you can deploy test automation and with Provar, that’s super quick. And then again, as we’ve already talked about, start monitoring the success or failure of that application or that deployment, using the metrics that you use to run your business. So the value that you realized as a consequence of deploying and investing in test automation.

Tristan Lombard: Yeah, I think that’s well said, and I also want to do a special shout-out to the University of Provar. It’s free and a great way to uplevel your skills also a shout out to our amazing customer success leaders and support. So, Andy, we want to end on a fun note, with a fun question here. Just wondering, in your opinion, what is the most underrated Provar feature and why?

Andy Menzies: Yeah. Look, I mean, at the end of the day, the support for metadata is by far the biggest feature set that drives kind of the minimum required capabilities of technology that realizes those positive business outcomes that I talked about. So I always try to take a feature set of technology and understand from a minimum required capability standpoint, is that, will drive the outcome that the business requires. And if I just think about that rather than a fun feature…it’s the metadata support. And I’ve seen, I’ve read, I’ve experienced people talking about how they are potentially able to do something similar and they can’t. So I’ve always identified with the fact that there are three different types of differentiation. Right?

There’s unique, there’s comparative and there’s holistic. So a unique differentiator is something that you can do that nobody else can do. You’ve probably invented it. You might even have a patent or patent for it. I believe that we have a whole heap of unique features in our technology. However, in lots of other companies, as is the case, when you come up with something unique that drives value, people start to copy your ideas, and that’s fine because if that’s delivering value to other customers, I’m happy with that. But by consequence, your unique differentiator becomes comparative. And 99.9% of deals have won on comparative differentiators.

I would urge anybody that wants to test Salesforce to think about Provar because this is our bread and butter. This is how we built the business. This is how we’ve scaled. This is how we have the velocity that we have. And this is why we have so many happy customers. Metadata support is my favorite feature, Tristan.

Tristan Lombard: This has been quite the interview y’all and I have to say I wouldn’t want to do it with anyone else except our CRO Andy, who has seen it all and will continue to be there to provide additional guidance. This is so much fun, Andy. I want to thank you so much for all that you do for our customers and everyone in the community. You’re quite a light, full of a wealth of information. We should do this again sometime.

Andy Menzies: Yeah, we should. Yeah, we can finish off what we started.

Tristan Lombard: With that said, I really, really appreciate all that you’ve done. And we’re going to do this again soon. Okay? From all of us here to you, take care and happy testing.

Andy Menzies: Thank you so much. Happy testing to you too. Bye.

Testing and test automation are key to improving the quality of your Salesforce releases. It is important to know the Salesforce test automation challenges in order our test automation efficient and effective. Learn how Provar’s key applications can help you scale your testing efforts here.