Secrets management and encryption
Provar honors the following principles with regard to passwords and secrets:
- Provar never exposes secrets in clear text in its UI or log files.
- Provar never transmits secrets in clear text over networks. This includes localhost connections.
- Provar does not rely on encryption keys that are built into the software (as these can be extracted by reverse compilation). These are always used in combination with a user-supplied secrets password that is used to salt the encryption. See this Wikipedia entry for additional information on encryption salting.
- Secrets passwords are never stored by Provar but instead supplied by the user when required.
Provar needs to store various secrets depending on the type of connection.
- Passwords are required for Salesforce, secured websites, Email and Database connections.
- Security Tokens and Client Secrets are required for certain types of Salesforce connection.
User account passwords are required to connect to third-party browser and mobile device hosting services such as Perfecto Mobile and SauceLabs.
Test Environments allow you to specify alternate details for connections in Provar:
- The Test Environment is built into the tool.
- Additional Test Environments can be defined in the Test Settings view.
When a new connection is created, details are always supplied for the Test Environment. Alternate details can then be supplied for specific Test Environments, if desired.
For example, a SalesUser connection can be defined that:
- Connects using UatSalesUser / password1 for the Test Environment
- Connects using ProdSalesUser / password2 for the PROD Test Environment.
At runtime, Provar uses any specific connection details for the active Test Environment, failing which the connection details are used.
Provar stores all secrets in centralized .secrets files. These are stored at the root of the Test Project and can either be encrypted or in plain text.
The main .secrets file contains all secrets for the Test Environment and also the secrets for Test Environments that do not have their own encryption passwords.
Where Test Environments have been encrypted with their own passwords, their secrets are stored in a separate .secrets file.
External secrets management
Some customers would prefer not to store their .secrets files in their Source Control Repositories (e.g. GitHub).
To accommodate this, Provar’s ANT task allows secrets to be supplied via specially named environment variables. This allows the secrets to be managed outside of Provar, for example, by the password plugin for Jenkins.
Provar allows secrets to be encrypted for entire Test Projects and also for individual Test Environments. This allows a PROD Test Environment, for example, to have its own secrets password that is only known to selected Provar users.
These approaches can be mixed and matched as follows:
- All secrets can be unencrypted. This is the default state for newly created Test Projects.
- The main .secrets file can be unencrypted and selected Test Environments encrypted with their own secrets passwords.
- The main .secrets file can be encrypted with the main secrets password and selected Test Environments encrypted with their own secrets passwords.
The encryption is done via the Jasypt open-source library using the user-supplied secrets password as the encryption salt:
- If the Test Project is encrypted then all secrets for the Test Environment are encrypted in the main .secrets file together with the secrets for any unencrypted Test Environments.
- If a Test Environment is encrypted then its secrets are encrypted in a separate .secrets file.
Secrets Password Requirements
Secrets passwords need to meet the following requirements.
- Must be at least 8 characters long
- Must contain an uppercase character
- Must contain a lowercase character
- Must contain a numeric character
- May contain special characters.
Newly created (and pre-existing) Test Projects are not encrypted by default:
The Encrypt Secrets context menu item against the Test Project’s entry in the Navigator view allows the Test Project to be encrypted.
After successfully Encrypt .secrets file looks like this with encrypted format.
The Encrypt context menu item against Test Environments in the Test Settings view allows individual Test Environments to be encrypted.
An entry will be created under test project file with the name of testenvironment.secrets.
Once encrypted, the following context menus are available against the Test Project’s entry in the Navigator view and Test Environments in the Test Settings view.
Change Secrets Password allows the secrets to be re-encrypted with a new password.
Change Secrets Password requires the existing password to be entered, as below:
For Test Environments, Unencrypt Secrets moves the secrets into the main .secrets file. For the main .secrets file, it decrypts the secrets and stores the .secrets file as plain text.
Unencrypt Secrets requires the existing password to be entered, as below:
Note that the .Secrets file comes in original format (un-encrypted format) with all the values except if there is any test environment encrypted value present.
Reset secrets encryption
For Test Environments, Reset Secrets Encryption moves all the secrets into the main .secrets file and sets them to empty values. For Test Projects, it removes the encryption and sets all secrets to empty values.
The existing password is not required for Reset Secrets Encryption.
Secrets password entry in desktop
When a Test Project is opened in Provar desktop:
- The user is prompted for the main secrets password if the Test Project is encrypted. Provar shuts down if no valid password is supplied after three attempts.
- If the pre-selected Test Environment (in Provar’s Settings Toolbar) is encrypted, the user is then prompted for the Test Environment’s secrets password. The Test Environment is automatically selected after three unsuccessful attempts.
Switching to encrypted test environments
The user is subsequently prompted for Test Environment passwords if:
- They select an encrypted Test Environment in the Settings Toolbar. The Test Environment is automatically selected after three unsuccessful attempts.
- They attempt to edit the alternate connection details for an encrypted Test Environment. The edit dialog does not open after three unsuccessful attempts.
- When they change the Test Environment on for an alternate connection to an encrypted one.
Invalid password attempts
Provar shuts down if the main secrets password is not supplied within 3 attempts.
- Provar automatically switches to the Test Environment if the password for an encrypted Test Environment is not supplied within three attempts.
- Provar does not allow alternate connection details to be modified if the password for an encrypted Test Environment is not supplied within three attempts.
- Provar prevents the Test Environment for alternate connection details being modified if the password for an encrypted Test Environment is not supplied within three attempts.
- Because Provar has no reliable way of storing invalid password attempt counters, these counters are reset when Test Projects are re-opened.
- The number of unsuccessful password attempts is tracked independently for each Test Environment until the Test Project is reopened.
- Secrets password entry under ANT
Main secrets password
The main secrets password can be supplied in two different ways when Provar is run via ANT:
- Via the secretsPassword property of the TestRunner task.
- Via the ProvarSecretsPassword environment variable.
Test environment secrets passwords
The secrets passwords for Test Environments can be supplied:
- Via the testEnvironmentSecretsPassword property of the TestRunner task.
- Via the ProvarSecretsPassword environment variable.
Invalid password processing
Provar validates the main and Test Environment passwords when the ANT task starts. If either password is invalid then:
- No tests are run
- An error outcome is returned.
- General information
- Licensing Provar
- Provar trial guide and extensions
- Using Provar
- API testing
- Behavior-driven development
- Creating and importing projects
- Creating test cases
- Custom table mapping
- Debugging tests
- Defining a namespace prefix on a connection
- Defining proxy settings
- Environment management
- Exporting test cases into a PDF
- Exporting test projects
- Managing test steps
- Namespace org testing
- Provar desktop
- Provar Test Builder
- Refresh and Recompile
- Reload Org Cache
- Running tests
- Searching Provar with find usages
- Secrets management and encryption
- Setup and teardown test cases
- Tags and Service Level Agreements (SLAs)
- Test cycles
- Test plans
- Testing browser options
- Tooltip testing
- Using the Test Palette
- Test Palette introduction
- Control test steps
- List compare
- Read test step
- String test steps
- UI Test Steps
- Using custom APIs
- Callable tests
- Data-driven testing
- Page objects
- Block locator strategies
- Introduction to XPaths
- Creating an XPath
- Label locator strategies
- Maintaining page objects
- Mapping non-Salesforce fields
- Page object operations
- Refresh and reselect field locators in Test Builder
- Using Java method annotations for custom objects
- Applications testing
- Database testing
- Document testing
- Email testing
- Mobile testing
- OrchestraCMS Testing
- Salesforce CPQ testing
- ServiceMax testing
- Skuid testing
- Vlocity testing
- Webservices testing
- Introduction to test scheduling
- Apache Ant
- Configuration for sending emails via the Provar Command Line Interface (CLI)
- Continuous integration
- Azure DevOps
- Running a Provar CI task in Azure DevOps
- Configuring the Provar secrets password in Microsoft Azure Pipelines
- Parallel execution in Microsoft Azure Pipelines using multiple build.xml files
- Parallel execution in Microsoft Azure Pipelines using targets
- Parallel execution in Microsoft Azure Pipelines using Test Plans
- Bitbucket Pipelines
- GitHub Actions
- Running a Provar CI task in GitHub Actions
- Remote Trigger in GitHub Actions
- Parameterization using Environment Variables in GitHub Actions
- Parallel Execution in GitHub Actions using multiple build.xml files
- Parallel Execution in GitHub Actions using Targets
- Parallel Execution in GitHub Actions using Test Plan
- Parallel Execution in GitHub Actions using Job Matrix
- GitLab CI
- Travis CI
- Execution Environment Security Configuration
- Parallel Execution
- Running Provar on Linux
- Salesforce DX
- Team foundation server
- Version control
- Zephyr Cloud and Server
- Salesforce testing
- Adding a Salesforce connection
- Assert Page Error Messages on Add/Edit Product
- Dynamic Forms
- Internationalization support
- List and table testing
- Salesforce Release Updates
- Salesforce Lightning Testing
- Salesforce Lightning Web Component (LWC) locator support
- Salesforce console testing
- Visualforce Testing
- Testing best practices
- Configurations and permissions
- Error messages
- Administrator has blocked access to client
- macOS Big Sur upgrade issue
- Resolving failed to create ChromeDriver error
- Resolving Jenkins license missing error
- Resolving metadata timeout errors
- Test execution fails – Firefox not installed
- Update to Opportunity field validation behaviour
- Licensing, installation and firewalls
- Test Builder and test cases
- Release notes