User Guide of the SoftComply Validation App for Confluence
SoftComply Validation for Confluence is an app that automates integrity checks of your Confluence instance in regular intervals – 1 x a week.
In the case of a test failure, SoftComply Support team will look into the reasons of the failure and notify the user when the validation tests can be re-run again.
Like all Atlassian Cloud apps, you can Trial the SoftComply Validation app for free for 30 days.
1. PREPARING FOR THE VALIDATION
Please follow the steps below to set up the Validation app prior to executing tests.
Alternatively, feel free to follow the video tutorial of the setup:
1.1. Check Global permissions
As an Admin user, log in to your Confluence site and go to Settings. In the left menu, open the Global Permissions view. Ensure that the site-admins or org-admins group has permissions to Access and Manage Spaces, as the Validation app creates a temporary space that the test user needs to access.
1.2. Create two test user accounts in Atlassian
The Validation app for Confluence includes tests on user permissions, verifying whether a user has access and editing rights to specific Confluence spaces and pages.
To set up these test users, follow the steps below:
1.2.1 Create Two Email Accounts
Use any email provider (e.g., Gmail) to create two new email accounts.
1.2.2 Sign Up for Atlassian Accounts
- Go to Atlassian Signup.
- Enter one of the email addresses created in Step 1.2.1, then click Sign Up.
- Complete the reCAPTCHA verification. You should see a notification about a verification email.
- Close the browser, reopen the signup link, and repeat the process with the second email address.
Note: Do not use email addresses already associated with existing Atlassian accounts.
1.3. Set up test users
1.3.1 Access User Management
As an Admin user, log in to your organization’s Confluence site.
Click the cogwheel icon in the upper-right corner to open Settings.
Select User Management from the menu.
1.3.2 Invite Test Users
Click the Invite users button.
Enter the email addresses of the test accounts created in Step 1.2.2.
Unselect all product roles to ensure the users do not receive unnecessary permissions.
Assign the User role for Confluence.
Click Invite people to send the invitations.
1.3.3 Assign Roles to Test Users
Open the Users list in User Management.
Locate one of the test users and designate them as the Admin Test User by assigning them to the site-admins or org-admins group.
Click the three-dot menu at the end of the user’s row.
Select Add user to group.
In the Group name field, type site-admins or org-admins.
Click Add groups to confirm.
The second test user, who only has the Confluence User role, will act as the Standard Test User.
1.4. Log in to Confluence
To prepare the two test users for the Validation app tests, you need to complete certain steps in Confluence for each user. For every step below, dismiss any wizard or intro pop-ups.
1.4.1 Verify Test User Accounts
Open the verification email for each test user in their respective mailboxes.
Follow the instructions in the email to complete the verification process for both accounts.
1.4.2 Set Up Atlassian Accounts
After clicking the Verify your email button in the verification email, the Atlassian account setup page will open.
For each test user:
Enter a full name.
Create and confirm a password.
Complete the setup process for both test users.
1.4.3 Update Email Visibility
After successfully logging in, open the Manage Account view.
Navigate to the Profile and Visibility page.
Scroll down to the Email Visibility section.
Set the test user’s email to visible to anyone.
Note: This step must be completed for both test users; otherwise, they will not appear on the Validation Settings page.
1.4.4 Enable Two-Step Verification
Navigate to the Security page in the Manage Account view.
Click on the Manage two-step verification link.
Enter the account password and click the Set up button.
Select “Authenticator app” as the verification method.
Follow these steps to complete the setup:
Scan the QR code using any authenticator app (e.g., Google Authenticator, Microsoft Authenticator).
Click on the icon to retrieve the security token data.
Copy and save the token securely, as it will be needed for the Validation Settings page.
Enter the 6-digit code generated by the authenticator app.
Click Connect phone to finalize the setup.
Save the recovery key by clicking on “Save options”, then click “Saved, let’s finish”.
Two-step verification should now be enabled.
1.4.5 Set Language to English
Navigate to the Account Preferences page.
Check that the language is set to English.
If needed, change the language to English to ensure compatibility with the automation framework.
Note: The automation framework currently supports only English.
1.4.6 Complete Initial Confluence Setup
Open the Confluence home page.
Dismiss any wizard or intro pop-up windows that appear.
Skip any questions related to user preferences inside Confluence.
1.5. Create a Jira project
Note: This step is optional and can be skipped if Jira is not in use.
1.5.1 Create a New Jira Project
Log into your organization’s Jira using the Admin Test User account.
Click on Create a new project.
Choose any available template.
Select Team-managed project as the project type.
Enter a name for the project.
Set the project access to Open or use Default permissions.
Click Create project.
Dismiss any guiding wizards that appear.
Note: If you are not using Jira, please skip this step altogether.
1.5.2 Create a Jira Issue (Optional)
Note: This step is optional and can be skipped if Jira is not in use.
Open the Jira project created in Step 1.5.1.
Click on Create issue.
Enter a summary and any relevant details for the issue.
Ensure that the issue does not contain any sensitive data, as the Validation app will capture a screenshot of the Jira issue macro displayed in the Validation report.
Click Create to finalize the issue.
Note: If you are not using Jira, please skip this step altogether.
1.6. Create a page in Confluence for Validation Documentation
Log into Confluence using the Admin Test User account.
Navigate to the space where you want to store Validation Reports.
Click Create to add a new page.
Enter a title for the page (e.g., Validation Reports).
This page will serve as the parent page for all Validation Reports generated by the Validation app.
Click Publish to save the page.
Copy the link to this page and add it to the Validation Settings page.
1.7. Add a Jira issue to a page (optional)
Note: This step is optional and can be skipped if Jira is not in use.
Depending on product, global, and space permissions, additional approval may be required to retrieve data from Jira. Access to Jira is necessary for testing the Jira Issues Macro in Confluence.
Steps to Add a Jira Issues Macro:
Open any Confluence page.
Click Edit to enter edit mode.
Click the “+” icon in the toolbar.
Search for Jira Issues Macro and select it.
If the search field is disabled and a “Login and Approve” notification appears, the Admin Test User must click the notification and approve the data retrieval.
Once approved, insert a Jira issue into the page.
Click Publish to save the changes.
2. CONFIGURATION OF THE APP
2.1 Open Confluence Validation settings page
As an Admin user, log into Confluence and navigate to the Confluence Validation page. This page can be accessed from the Confluence Settings view.
Note: Confluence Admin rights are required to configure and run the Validation app.
2.2. Select a Reporter
In the Validation Settings page, choose a user to act as the Reporter for Validation reports.
The selected user will be displayed as the reporter on the Validation Report page.
Only users with a public email will appear in the selection list.
Important Notes:
The Validation app communicates with Confluence via the direct REST API to generate and store Validation Reports.
If you are not the Reporter, the designated user must generate an access token for the Validation Report and share it with you, so you can add it on their behalf.
2.3. Provide an Atlassian access token
To allow the Validation app to generate the Validation Report in your Confluence instance, you must provide an Atlassian access token.
Key Requirements:
The access token must be generated by the same user selected as the Reporter.
If the Reporter is changed, a new token must be created.
Important Notes:
The Atlassian access token does not change when you update your password.
You can revoke the access token at any time, which will immediately prevent the Validation app from accessing your Atlassian account to generate reports.
2.4. Select a Validation Report page
Choose a location for storing Validation Reports.
Only pages within spaces where the Reporter has access can be selected.
Select the page created in Step 1.6 (Validation Reports page).
This page will act as the parent page for all newly generated Validation Reports.
2.5. Add Test Users
Select a Test Admin User
Choose the test user who is a member of the site-admins or org-admins group.
Enter this user’s Atlassian account password.
If two-step verification (2FA) is enabled, enter the 2FA token as well. This can be found from Step 1.4.5.
This field only displays users with the Confluence Admin role.
Select a Standard Test User
Choose the second test user who has the Confluence User role.
Enter this user’s Atlassian account password and 2FA token (if enabled).
This field only displays users with the Confluence User role.
Important Notes:
Only users with a public email are visible in the dropdown lists.
If a test user is missing from the list, follow Step 1.4.3 to make the email public.
2.6. Add Jira Issue (Optional)
Note: This step is optional and can be skipped if Jira is not in use.
If you want to test Confluence’s integration with Jira, provide a Jira issue ID.
This should be the ID of the Jira ticket created in Step 1.5.2 (e.g., TV-1).
The Validation app will use this issue to verify that Confluence can successfully fetch and display a Jira Issues Macro.
2.7. Save Configuration
Ensure all required fields are filled out.
Click the Save Configuration button to apply the settings.
Refresh the browser to confirm that the configuration has been saved successfully.
3. AUTOMATING TEST EXECUTION
You can perform a manual validation of your Confluence Cloud instance once a week or set up automatic test execution on a recurring schedule.
Important Notes:
Once automatic test execution is enabled, manual test execution will no longer be available.
3.1 Steps to Set Up Automatic Test Execution
Navigate to the Validation Settings page.
Enable the scheduler by switching the toggle on (green).
Select a weekday for the tests to run automatically.
Once configured, the Validation app will execute tests weekly at the scheduled time.
3.2 Enable Notifications for Automated Validation
To receive notifications upon the completion of automated validation:
Select a Reporter from the dropdown menu.
The chosen user will receive an email notification after the validation is completed.
The email will contain a link to the validation documents for review.
4. VALIDATION RESULTS
4.1. Viewing Test Execution Details
The Validation app page provides an overview of all executed validation runs, including:
Total number of test cases executed
Completed and failed tests
Validation run status
Timestamps of execution
How to Access Validation Results:
Click the Apps menu in Confluence.
Select Validation for Confluence.
Alternatively, navigate to the page via the Validation Settings page.
4.2 Understanding the “Details” Column
In the Validation app page, the Details column provides insights into the test execution status:
If the Validation Report has been successfully generated, the column will display a link to the report.
If there was a misconfiguration, preventing the report from being generated, it will display “Tests run has been started” instead.
A test execution is considered complete when the status shows “Completed”.
4.3 Test Execution and Report Statuses
Test Execution Statuses:
COMPLETED (with failed tests) – The test execution finished, but some test cases failed.
COMPLETED (all tests passed) – The test execution finished successfully, and all test cases passed.
SCHEDULED – The test execution is queued and will start shortly.
UNKNOWN – A failure occurred, preventing the tests from being executed.
EXECUTING – The test run is in progress.
Validation Report Statuses:
Validation Report generated – The report was successfully created.
Tests run has been started – A failure occurred, preventing the report from being generated.
Failed to generate Validation Report – The report was not generated, likely due to an invalid access token.
4.4. Viewing the Validation Documents
You can access the Validation Documents by clicking the View Report button on the Validation app page.
When is the View Report Button Active?
The test execution must be completed.
The Details column must display “Validation Report generated”.
Once active, clicking the View Report button will open the generated Validation Report for review.
4.5 Important Notes on Validation Reports
Opening a Validation Report for the first time may take several minutes, as the reports contain high-definition images that impact load time.
Documents Generated per Test Run:
With each test run, the following four documents are generated:
Master Validation Plan
A comprehensive document outlining the validation approach and a risk assessment of the features.
Validation Report
A detailed report listing all test cases, including their steps, screenshots, and statuses.
Test Protocol
A structured document containing a list of test cases and their corresponding steps.
Master Validation Report
A high-level report summarizing the overall outcome of the test run.
5. BOOK A DEMO
In case you would like to learn more about the Validation app, you are most welcome to schedule a LIVE DEMO of the app with the SoftComply team.
