The Agile Testing Pyramid

Overview: Agile Testing is a testing practice that works on principles of agile methodology. Unlike the traditional sequential models like Waterfall method, Agile Testing begins as soon as development begins. Software Testers are part of agile teams (and not different team) and contribute towards building the software as per the customer requirements.

In a nutshell, testing in agile is not standalone or a separate phase.

For an overview, below are the testing activities in Agile model:

1. Release Planning: It is done

• For every Iteration
• For Testing Activities during an Iteration.

2. Regression Testing

The Agile Testing Activities during an iteration include −

• Participating in iteration planning
• Estimating tasks from the view of testing
• Writing test cases using the feature description
• Integration Testing (End to end testing for various stories developed in different sprints)
• Functional Testing (based on Feature /Stories)
• Defect Tracking/ Fixing/Retesting
• Acceptance Testing (Show and tell sessions)
• Test Status Reporting
• Participating in retrospective meetings.

The Agile Testing Pyramid

The Agile Testing Pyramid was introduced by Mike Cohn. It explains the contrast between testing in Waterfall and iterative models (Agile). The differences are important for Agile success and require mindset change across organization.

In the Traditional Testing methods (read Waterfall), Majority of testing is done using tests initiated through the User Interface. There is some automation, for example for regression tests. Also, there is some level of unit testing by developers (usually less coverage is seen) done before code is handed over to QC for testing.

We can say that traditional testing is majorly designed to “find bugs”. Basically, developers write code to implement features and then QA engineers slog on it to find the problems, logging defects for the developers to fix towards the end of project life cycle.

However, Things have changed with modern approach of Agile testing. In Agile, we have a pyramid with a bigger base of Unit Tests i,e developers need to contribute to Quality too (change in mindset).

Also, Agile testing relies more on automation. In the nutshell, we can say that, approach in agile is basically to "prevent bugs" rather than to "fix bugs" as in traditional methods.

Fig -I Agile Testing Pyramid

Fig- II Traditional Testing Pyramid

The Agile pyramid requires mindset change in organizations and different way of using the existing skills. Developers test more with unit tests. A commonly used framework for this is Junit (for Java based applications). There is a difference in approach for testing example- Test Driven Development. QA teams also rely more on automation than on Manual testing.

Mature Agile teams create automated tests directly from user stories. That is, both unit and functional tests tend to be automated. We can validate the correctness of application under test on a continuous basis by running our automated tests many times a day (Completely automated DevOps pipeline is required to achieve this).

For automated testing in the Acceptance Test layer, a design approach called as "The Thin UI" can be utilized. This approach ensures that all logic is moved to the logic layer (usually UI is not developed until the end), leaving just the UI controls that send and receive messages. By doing this, test suite can test the logic without going through the UI. They just communicate with the logic layer to test everything in and below it. Example - API testing, Services testing.

In many projects, automated SOAP tests are created to validate the underlying functionalities. Many times, UI testing is even outsourced to a smaller organization outside the main organization doing testing project.

Another aspect of automation is Regression tests. When developing iteratively, regression testing is a must. New builds are deployed daily or may be many times a day. It is essential to know that the new builds did not break existing functionalities, so everything needs to be tested continuously- probably every day. This is very difficult to achieve manually. It also takes longer and longer in the later sprints where we have more regression to do.

Benefits of Automating in Agile (Using Thin UI approach)

• Automation gives us benefit of improving overall team efficiency and defect finding capacity.
• With automated tests we get high code coverage and we can easily identify when a change has broken an existing functionality by going to the test that is failing, see what code it tests and quickly make the required fix. This can be attributed to larger base of Unit tests in Agile Testing pyramid
• When most of the testing is automated, the amount of test work can track linearly with story development, some old tests may still have to change as new stories cause changes in old stories.
• By automating what we know must be true, we have constant validation from regression testing and fewer surprises. Please note that there may still be some unanticipated behaviors from the overall complexity of the integrated systems and environmental issues.

Please note that automating from UI layer may not be equivalent to automating the lower level tests as UI tests are more fragile because the UI structure (Example- Web elements) changes over time.

Why should Agile Testing Pyramid be used?

• Agile process emphasizes speed and efficacy. The testing pyramid streamlines the testing process. The pyramid is built to run the easiest tests at the beginning, testers manage time better, get better results and essentially make life easier for everyone involved.
• The pyramid provides testers with the right priorities. If test scripts are written with a greater focus on the UI, then chances are that underlying business logic and back-end functionality is not sufficiently tested hence affecting product quality and more leaked bugs. By implementing the pyramid, such situations are avoided.

Conclusion: Agile testing done in right way will give your team a firm foundation for iterative development. We can be confident on what is working, thanks to continuous testing. However, making teams mature in Agile way of working takes a longer time than usually thought. Often, it is about mindset shift.

*********

Software testing blog by leading job portal for software testers in India - Qualitician. Bringing the latest news, articles and tutorials for software testing professionals.