Articles in this section

See more

Software IDM Testing - Putting it all together

Avatar
Ian Bassi (SIDM)
  • Created
  • Updated
Follow

Content

Introduction

This document goes through the process of testing at Software IDM, starting at the beginning capturing the User Stories and Acceptance criteria, all the way through to documenting and reporting on the testing that has being carried out.

No previous knowledge of testing is required to follow this guide, and if there any are questions or queries, or you would like a demo of anything discuss in these articles, please schedule a meeting with me here where we can discuss it further.

For more information on why you would want to test, see the article below:

Why Test? Problems with Testing, How we can help.

Approach

User Stories

Once the high level documentation for the solution has been created, we can use this to create User Stories.  User Stories break the work down in the smaller chunks, making it more manageable, and allowing us to track progress and make it easier to test.

Creating User Stories

Acceptance Criteria

Once we have the User Stories, we can then define the Acceptance Criteria for the user stories.  It is okay to create draft acceptance criteria by reviewing the High Level Design, but these should be reviewed as a team, with the Key Stakeholders signing the Acceptance Criteria off after the meeting. This is a critical step, as it sets the target in stone, and avoids time being lost when test cases are written, and then need to be changed because the Acceptance Criteria change.

Once the Acceptance Criteria have been agreed, they can be changed, but a change request will need to be raised to cover the cost of changing the previously agreed Acceptance Criteria.  If the Acceptance Criteria for a User Story can not be agreed, that User Story should be pushed back to a later sprint, where the Acceptance Criteria can be agreed.

Capturing Acceptance Criteria

Test Cases - Azure Dev Ops

Once the Acceptance Criteria has been signed off, the Test Cases can be written in Azure Dev Ops.  This can be considered the design stage for the test cases to be written in test panel.  Once the test cases have been written in Azure Dev Ops, these should be reviewed and signed off to ensure they are correctly covering all the acceptance criteria.

Writing Test Cases

Test Suites - Azure Dev Ops

Once the test cases have been written, the Test Suites can be created in Azure Dev Ops.  This allows the tests to be assigned to appropriate suites, and assigned to the relevant users to complete the testing.

Creating Test Suites

Test Panel - Initial Setup

Once the Test Cases have been created in Azure Dev Ops, we can now prepare Test Panel within Identity Panel.  All the providers which are going to be used should be added to the solution before Test Panel is setup. With the approach taken by Software IDM to have separate engineers complete the role of Developer and Tester, it is okay for development to start before the automated testing has being setup.

Test Panel - Initial Setup

Test Panel - Creating Fixtures

Once Test Panel is setup, we can then start creating the fixtures which are going to be required.

Test Panel - Creating my first fixture

Test Panel - Creating Assertions

Once the fixtures have been created, we can then start creating the assertions which are going to be used.

Test Panel - Creating my first assertion

Test Panel - Creating Tests

Once the fixtures and assertions have been created, the test cases can be created.

Test Panel - Creating my first test

Linking Test Panel and Azure Dev Ops

To help support the team building the solution, adding links between Test Panel and Azure Dev Ops will make maintaining both systems easier, and allow tests to be run quicker.

Linking Test Panel and Azure DevOps

Development

Development can now be started, if it has not already being.  The key regression tests can be setup to run nightly.  Initially these tests will fail, but as development begins, the tests will start to pass, and its is a great way to track progress.

If a new feature breaks an existing part of the solution, by having the tests running on a schedule, the engineer who is doing the development will be aware of what has broken, and it will be fresh in their mind what they were working on and what could have caused this issue. This saves a significant amount of time, compared to only doing regression testing after development completes, because trying to remember what changes you might have made four weeks ago, is a lot more time consuming, than remember what was changed yesterday.

Test Panel - Executing Tests

Once development has started, if additional testing is required, the tests can be run manually. The following article describes this process.

Test Panel - Running my first test

Azure Dev Ops - Executing Tests and Raising Defects

While Test Panel allows the tests to be automated, it does not provide a way to document each step, and raise defects. Azure Dev Ops allows us to do this in great detail, providing the engineer with the information required to further investigate the issue, and fix the defect.

Executing Test Cases and Raising Defects

Reporting - Test Panel

Being able to automate the testing saves a lot of time, but being able to report on the test outcomes allows the progress to be tracked, and spot any trends early on.  The article below provides example reports that can be installed.

Test Panel - Running my first report on testing

Reporting - Azure Dev Ops

Being able to provide evidence to key stakeholders and change review boards can great increase confidence in upcoming go-lives.  Azure Dev Ops allows us to create details reports, which contains links to all the evidence of the testing which has been carried out.

Azure Dev Ops - Test Case Reporting

Summary

In summary, the SoftwareIDM Test Driven approach to solution development bring significant benefits to customer solutions, from reducing the amount of time required, increasing the quality of what is being delivered, and ensure the customer is getting exactly what they want.

For each test case that is written, it can be used in the future for regression testing, and with the automation in Test Panel, it takes minimal time for the engineer to start the testing process.

The evidence captured in Azure Dev Ops provides a great training tool for on-boarding new members of the team, as the User Story and Acceptance Criteria provide the details on what the solution should be doing, and the Test cases provide the step by step instructions on how to run the different processes.

Related articles

Comments

0 comments

Please sign in to leave a comment.