Now that the Jenkins server is set up for Provar test execution, we can begin setting up the Salesforce org with the Flosum installation to queue the Jenkins job that we just created.
Prior to setting up the webhook to trigger our Provar job on another end system, a pipeline has to be created in Flosum. Before you can initiate a pipeline, you need to have a valid branch in Flosum to run it from. The entire process is detailed in the following steps.
- Registering your Organization
- Creating a Snapshot
- Committing to Branch
- Creating a Webhook
- Creating a Pipeline
Registering your Organization
The first step in setting up the pipeline in Flosum is to add an organization. Navigate to the Orgs tab in your Flosum org and click New.
Enter the Organization Name, Authentication Type (automatic is easier), and Organization Type. This needs to be an org that you are connecting to from Flosum to trigger the pipeline. In practice, this would be whatever org you are making deployments/triggering changes on that triggers your Provar tests elsewhere. In other words, if you want to run Provar tests following a deployment/change in a Developer org, your setup might look like the screenshot given below.
Once you enter these field details in, you’ll have to authorize Flosum to connect to this org by clicking Authorize in the top right corner and following the Salesforce authentication there.
Success! Your org has been added to Flosum and we can continue on to the next step of creating a snapshot.
Creating a Snapshot
In order to create a snapshot, navigate to the org we just created and select Create Snapshot.
First, you will enter a name for your Snapshot. A snapshot is essentially a specific capture of metadata that you wish to deploy or validate. You can create a partial or complete snapshot. You can configure a lot of options for the snapshot.
For example, you can select the number of days that you want to retrieve changes for the selected components. You can also filter the components based on component type, component name, or changed by fields.
The operator can be set to a wide variety of options, depending on which components you are trying to filter for.
Once you’ve selected a field and an operator, you can use the search icon on the far right to search for specific components by value. You can add each individual component you want by checking the boxes next to the relevant item.
For testing purposes, we’ve kept it simple and changed the retrieval date to ‘0’ days to retrieve all components.
Once you are satisfied with the snapshot, click create snapshot in the top right corner.
Give Flosum a minute or so to complete the snapshot process. You can track the status by refreshing the snapshot screen and checking the Is Completed? checkbox. Additionally, you’ll want to make sure that the No. of Metadata Items is greater than ‘0’.
That’s it! Your snapshot is created from the Org we registered previously and we are ready to commit it.
Committing to Branch
Once you are on the snapshot screen, the commit to branch step will open the next screen to select all components you’d like to commit.
A note here that all of this is being committed to the Flosum’s native Version Control System, and not your personal/company’s repository. It’s a great feature for storing all of this metadata information directly in your Flosum org and not having to set up your own Git repository just for this process.
If you do not have an existing branch in Flosum (which you can check by selecting the search icon next to the Commit to Branch field), then you can enter a name in the Branch Name field and Flosum will automatically create that branch for you.
Next, you will select all of the components from the snapshot that you’d like to commit to the branch.
From there, select Continue Commit to Branch.
On the next screen, you need to verify all of the components you’d like to commit based on which components are considered safe to commit.
In this example, we’ve selected everything from our org snapshot. Once you’re happy with it, select Continue Commit to Branch to officially commit these components to the branch in Flosum.
It will take Flosum a minute to process the commit, and then it will navigate you to the Branch view as shown below.
Now that you have a branch, we can begin setting up your Flosum Pipeline.
Creating a Webhook
To create a webhook in Flosum, navigate to the Settings tab and select webhooks.
Click Add Webhook and enter in the Webhook Name, URL, and your Jenkins authentication. For the Jenkins authentication, the password must be replaced with a valid Jenkins API Token generated by the same user you are using to authenticate. This has to be set up properly from the Jenkins server prior to adding the webhook here.
The URL needs to be the build URL pointing to a specific Jenkins job. Please see the example given below.
Jenkins URL: https://demojenkins.com:8080
Job Name: FlosumTrigger
Build Parameters: TEST_PLAN=Regression, BUILD_FILE=build.xml
API Token Name: FlosumToken
Then the webhook URL would be the following:
You can also find this URL underneath the Build Triggers section on your Jenkins job.
A reminder here that the URL of the Jenkins server must be reachable from your Salesforce Org. If your Jenkins server is behind a proxy or firewall, then it needs to be opened up to allow traffic from Salesforce IP ranges.
Also, remember to check the Wait callback response from service to wait for the Jenkins webhook to return successfully before continuing to the next Pipeline step.
Now that the webhook is created, you can continue on to the pipeline creation.
Creating a Pipeline
To create a Pipeline in Flosum, navigate to the Pipelines tab and select New.
Enter the Pipeline Name, and leave the rest as the default.
Next, you need to add some Pipeline steps. Select Pipeline Steps and once it loads, click New.
We’ve named our webhook Jenkins, so this is what the step looks like.
Since there is no next step here, the On Pass and On Fail fields can be left as Stop for now.
For the purposes of this integration, our pipeline only has a single step that will trigger our webhook to kick off our Provar tests. For a practical pipeline, you would have multiple pipeline steps that include deployment, validation, and rollback.
Once you’ve added all of the necessary Pipeline steps, click Save Steps.
The Pipeline is set up and is ready to be queued from the Branch we created earlier. Navigate to the Branch view screen for the branch we committed to earlier. From the branch, we can run the Pipeline directly.
Select the Pipeline you just created.
You can view the Pipeline results from the Pipeline Executions related list from the Pipeline view, or you can view the Deployment History from the branch view.
If you navigate to the Deployment History >Metadata Logs you’ll see the result of each Pipeline execution.
Navigate to one of these Metadata Logs, and the Related tab will show any relevant error logs in the Notes & Attachments related list.
Note: This webhook step does not wait for the Jenkins job itself to complete, but rather just the callback response to return. Currently, there is an error in Flosum that shows the deployment step as failed if the callback response is a 201 (which is what Jenkins returns when successfully triggering a job).
This will be the attached log, and the deployment history and pipeline step execution will both show as failed, even though the call was successful.
If you are queuing your pipeline and do not see anything occur on the Jenkins side, then you might be getting a timeout due to a network firewall blocking the connection. If it is blocked, you can check the deployment history metadata logs for a similar message.
You may also see the following error if your webhook URL is malformed or not sending properly: webhook error: Read timed out.
For a more detailed view of the results of the tests, you’ll have to navigate to the Jenkins job itself. You can also configure your build.xml in Provar to email test reports once completed.
- General information
- Licensing Provar
- Provar trial guide and extensions
- Using Provar
- API testing
- Behavior-driven development
- Creating and importing projects
- Creating test cases
- Custom table mapping
- Debugging tests
- Defining a namespace prefix on a connection
- Defining proxy settings
- Environment management
- Exporting test cases into a PDF
- Exporting test projects
- Override auto-retry for Test Step
- Managing test steps
- Namespace org testing
- Provar desktop
- Provar Test Builder
- Refresh and Recompile
- Reload Org Cache
- Running tests
- Searching Provar with find usages
- Secrets management and encryption
- Setup and teardown test cases
- Tags and Service Level Agreements (SLAs)
- Test cycles
- Test plans
- Testing browser options
- Tooltip testing
- Using the Test Palette
- Test Palette introduction
- Control test steps
- Generate Test Case
- List compare
- Page Object Cleaner
- Read test step
- String test steps
- UI Test Steps
- Using custom APIs
- Callable tests
- Data-driven testing
- Page objects
- Block locator strategies
- Introduction to XPaths
- Creating an XPath
- Label locator strategies
- Maintaining page objects
- Mapping non-Salesforce fields
- Page object operations
- Refresh and reselect field locators in Test Builder
- Using Java method annotations for custom objects
- Applications testing
- Database testing
- Document testing
- Email testing
- Mobile testing
- OrchestraCMS Testing
- Guide in Salesforce CPQ Testing in Provar
- Guide in ServiceMax Testing
- Skuid Testing
- Vlocity API Testing
- Webservices testing
- Introduction to test scheduling
- Apache Ant
- Configuration for Sending Emails via the Provar Command Line Interface
- Continuous integration
- AutoRABIT Salesforce DevOps in Provar Test
- Azure DevOps
- Running a Provar CI Task in Azure DevOps Pipelines
- Configuring the Provar secrets password in Microsoft Azure Pipelines
- Parallel Execution in Microsoft Azure Pipelines Using Multiple build.xml Files
- Parallel Execution in Microsoft Azure Pipelines using Targets
- Parallel execution in Microsoft Azure Pipelines using Test Plans
- Bitbucket Pipelines
- Gearset DevOps CI/CD
- GitHub Actions
- Integrating GitHub Actions CI to Run Provar CI Task
- Remote Trigger in GitHub Actions
- Parameterization using Environment Variables in GitHub Actions
- Parallel Execution in GitHub Actions using Multiple build.xml Files
- Parallel Execution in GitHub Actions using Targets
- Parallel Execution in GitHub Actions using Test Plan
- Parallel Execution in GitHub Actions using Job Matrix
- GitLab Continuous Integration
- Travis CI
- Execution Environment Security Configuration
- Provar Jenkins Plugin
- Parallel Execution
- Running Provar on Linux
- Salesforce DX
- Team foundation server
- Version control
- Salesforce testing
- Adding a Salesforce connection
- Assert Page Error Messages on Add/Edit Product
- Dynamic Forms
- Internationalization support
- List and table testing
- Salesforce Release Updates
- Salesforce Lightning Testing
- Salesforce Lightning Web Component (LWC) locator support
- Salesforce console testing
- Visualforce Testing
- Performance Best Practices
- Testing best practices
- Configurations and permissions
- Error messages
- Administrator has blocked access to client
- macOS Big Sur Upgrade
- Resolving failed to create ChromeDriver error
- Resolving Jenkins license missing error
- Resolving metadata timeout errors
- Test execution fails – Firefox not installed
- Update to Opportunity field validation behaviour
- Licensing, installation and firewalls
- Test Builder and test cases
- Release notes