By: Richard Clark, CTO at Provar
The most important part of testing is actually about putting information about test results to use.
When data is shared with the right stakeholders at the right time and in the right way, the value of that information increases exponentially. By integrating Provar with your collaboration platform, like Slack, your team can find and fix bugs more quickly and raise the visibility of test results as soon as they are received.
Accelerate your feedback loop with Provar real-time test results on Slack
Slack is a team collaboration application that can integrate with many external applications and Provar is no exception. Using incoming webhooks, external apps can automatically post messages to Slack channels when specific events under their control occur. Using Slash Commands, users can query or update information in external apps from within the Slack UI. You can even set up bots to listen to Slack conversations and automatically take actions in response. These integrations allow you to turn Slack into a centralized real-time messaging hub across multiple systems.
It’s already very easy to post to Slack as part of a CI/CD pipeline to include the complete test results but sometimes you may want to know about a failure well before you get to the end of the test execution. (Be sure to check out this customer success story that profiles how Canada Drives was able to do just that and improve the visibility of automation run reports using Slack.)
“By integrating Slack into the Jenkins jobs we get all the automation runs reported to a Slack channel, including pass or fail details. This keeps devs, admins and testers informed of progress.” – Chandra Alahari, QA Automation Engineer at Canada Drives
How to implement Slack real-time messaging for Provar test results
In this article, I demonstrate a simple integration scenario between Provar and Slack to provide a mechanism to post messages into a Slack channel.
What does that mean? By integrating Provar with Slack, you can use a dedicated Slack channel to automatically notify internal team members about important test results in real-time and you can define the type of test results that you want to share. In general this would typically include notifications such as when a test step or test case fails and/or when a test run has been completed.
To create a Provar + Slack integration, you need Slack admin rights for your Slack workspace. I recommend that you create a new Slack workspace for the purpose of developing your integration so that you don’t clutter your production Slack workspace with your unit test results. (You only need to change one value to connect it to your target workspace.)
Step 1: Create a webhook
Incoming webhooks are a simple way to post messages from external sources (in this case, Provar) into Slack. They make use of normal HTTP requests with a JSON payload, which includes the message and a few other optional details. You can even include message attachments to display richly-formatted messages.
I’m going to use the same webhook for all of my examples. If you want to post messages to different channels, repeat the exercise for each channel you want to publish to and make a careful note of each webhook URL.
Steps for how to create a webhook are documented by Slack here.
Above: View of Slack webhook settings. Make sure you copy the webhook URL.
The webhook URL is automatically generated and unique to you. Go ahead and try out the curl command first so you know it’s working.
I added a custom username for the integration and uploaded a copy of the Provar logo.
Step 2: Add a new Provar connection
Provar collates all connection details so that you can have a single repository of system connections. Your new connection should be very similar, if not identical to this:
Above: View of Slack connection settings in Provar.
The connection defines the web service endpoint and authentication. In my example, I do not need authentication since the webhook that we created above contains the service endpoint and we haven’t restricted it.
Note: Always use HTTPS to secure the request parameters and the full URL.
Step 3: Create a Provar callable test
The next step is to create a simple Provar callable test that is capable of posting a message to the webhook URL. This allows us to pass result parameters and have a single utility test to maintain. It also means that your team can leverage the Provar + Slack integration without having to understand how to do the integration on their own.
You can create this yourself following the steps below:
(1) Create a new test case in Provar, call it SlackMessage. Do not start the Test Builder.
(2) Add a string Input Parameter and set your test to be callable. I’ve named my parameter ipMessage (the ip prefix is a naming standard I use so I know it’s an input parameter rather than a local variable).
Above: View of Provar test case input parameter details and the test case settings.
(3) Add the newly created Slack connection to your SlackMessage test case.
(4) Add a REST web service step from the Test Palette and configure it to point to the webhook that you created in Step 1. The resource URL should be a relative path to the connection you created, i.e. everything after https://hooks.slack.com/services in the webhook URL. You should store this as an environment variable for security purposes.
Make sure you set the Content-Type in the request header and JSON payload for the body including your input parameter, as shown below:
Above: View of web request step parameters.
(5) Run your new callable test case and check everything is working before we start including it everywhere.
Pièce de résistance!
Now you have a working solution for posting messages to Slack from Provar, give yourself a pat on the back. All you need to do is drag and drop your callable Slack message test as a step into your other test cases and set the message parameter of what you want to write. Here are a few of mine:
A teardown test that logs all test run errors automatically in a single API call. Using teardown means you don’t have to change any existing tests. To test this I wanted a test that fails on purpose using a Fail Test step:
Alternatively, use a Finally step to automatically log the result of a specific test. You may want to make this conditional on failure only to be less chatty!
Integrating Slack with Provar
This is just one of the many ways that you can integrate Provar with other applications to enhance your test reporting. If you’d like to learn more about these and other strategies you can use to make the most out of Provar, you can refer to this case study highlighting a Provar customer, Scaling Salesforce test automation with Canada Drives.
If you want to learn more about writing Slack integrations, including tips on formatting your JSON payload, then there’s a nice short Trailhead Module from Salesforce here, and there’s a great article by Christophe Coenraets here about integrating Slack using Invocable Apex and Process Builder if you’re a Salesforce Consultant.
If you want to test out your message formatting and payload string then use the Slack API Message Formatter to interactively test your string values quickly rather than keep running a test every time.