Common challenges to Software Testing in Agile Development

Overview: In traditional practices of project development, Software Testing used to be carried out towards the end of project cycle. Often, due to spill over from other development activities, testing duration used to be squeezed to bare minimum resulting in faulty code delivered to production. Agile methodology is different.

Agile Testing is a testing practice that works on principles of agile methodology. In Agile methodology, unlike the traditional methods, development and software testing are carried out iteratively i.e. each iteration of code built is also tested within the same iteration.

Agile teams are self-sustaining units i.e. they cannot depend on external teams for testing the code. Hence, the teams are generally a mix of developers, testers, and business analyst. Everyone in agile team is responsible for quality of code delivered and not just testers.

Software Testing Activities in Agile Projects

1. Iteration 0 − In Agile terminology, Iteration 0 is usually the planning sprint which sets the tone for the project. In this sprint, from testing perspective, team works on

• Outlining the key requirements and use cases that will drive the scoping
• Preparing the high-level testing plan
• Identifying and outlining high-level testing scope. (More importantly, what is not in scope of the agile team)
• Identifying the tools, skills required, and resources required.

2. Post the planning sprint, "Construction" Iterations begin. Construction Iterations are nothing but the actual development/test cycles where a shippable product is being built.

In this Phase, testing team goes through below work steps:

• Participating in iteration planning
• Estimating tasks from testing perspective
• Writing test cases using the user stories
• Test Execution - Functional and regression testing
• Defect logging/Triaging/Retesting
• Test Status reporting
• Participating in retrospective meetings

The above-mentioned steps are repeated in every iteration until a release is deployed into Production.

3. Usually the iteration testing is followed by end-to-end system testing from user perspective, called Acceptance Testing. This phase will cover end to end flows developed across different agile teams.

Challenges of Testing in Agile Methodology

1. Frequently changing requirements: The very essence of agile is that project should be able to adapt to changing customer needs. Most projects which run in agile methodology are basically evolving and change of priorities and plans is quite possible. Due to this change in requirements, a lot of work already done may become redundant. For example, test cases, automation suites etc.

• Since the change is inevitable, testing teams must learn to adapt to the change quickly. Sometimes, the change comes in towards the end of sprint and is quite big to accommodate within sprint timelines. In such cases, tester should be able to communicate clearly to stakeholders- what areas are impacted by the change, which tests are executed and what cannot be covered in current sprint. This helps stakeholders to take an informed decision to take the change to production or not.
• The usual practice is to carry out a risk-based testing around the change.

2. Inclination towards Development teams: If the Agile teams are not mature enough, Story points are not shared considering the testing impact. For example, If the proposed change in sprint is a minor one from development perspective, but the change is replicated in many pages of application under test resulting in higher test effort. In such scenarios, it has been observed that testing teams gets pushed back to reduce effort so that overall velocity is increased.

• In such scenarios, it is the responsibility of Scrum Master/Agile coach to guide the team and let the appropriate effort be shared considering both development and testing.

3. Running Agile sprint in a Waterfall way: Many times, with less mature agile teams, the development work takes most of the time duration on sprint to finish and testers are left with 2-3 days towards the end of sprint. In such cases, software testers often struggle to finish the agreed upon scope for testing within the iteration. This happens due to incorrect estimation for development tasks.

• The Scrum Master/Agile coach should be able to guide the team to perform continuous testing. Software testers should be involved in story development from the beginning
• Also, Pair development can be practiced i.e., a Developer and Tester can work together on one story. The story is called ‘done’ only when both development and testing are complete for the story.
• While the developer is working on the code, tester can work on identifying the test conditions and preparing the data.
• The stories can also be tested incrementally i.e., whatever is developed is tested alongside (with or without pairing in team) and a feedback can be sought from product owner. This will ensure that software testers are getting adequate time to test, and team is moving together.

4. Automation Skills: Many times, it is seen that to move to agile, the team structure and skills are ignored. For example, relying on manual testing for testing. Agile without automated suites is either not successful or burns out resources. Projects move to agile as time to market is an important criterion, hence there is no alternative to automation. Upskilling automation skills is one of the hardest challenges for testers.

• Upskilling team with automation skills requires lot of patience. Hence, Software Testers should be trained in a time bound training program related to test automation frameworks, TDD/ATDD, API tests etc.
• Training plans should have full management support. Management should not expect immediate results and ensure it is a gradual process to change.

5. Automation Strategy (or Lack of it): Many a times management thinks that automation equates to just creating scripts. However, it is far from true. Automation requires strategizing end to end from CI/CD pipeline to creation of a test suite which runs lights out. Poor strategy can slow down the testing and hence defeating the very purpose of agile. Many times, in projects, each phase from development to deployment has a different automation strategy and each is in silo from the other. For agile projects to be successful, automation should preferably be integrated end to end.

• To avoid high risks to project functioning, it is advisable to start the automation by doing a small Proof of Concept for tool selection on the application under test.
• Tools should be selected considering long term goals of project, License costs and compatibility with the technology of application under test. For example- Tools good for testing SAP applications might not fit for testing mobile apps.
• Once the tools are selected, a gradual transition should be made to full adoption of tool.

6. Continuous Regression: Agile projects require continuous regression within the sprints to ensure new build did not break already working code. The need to continuously regression test adds to efforts within the sprint. Often, due to lack of time in the sprint, regression is ignored, and it leads to defects being introduced which are not caught until it is quite late in the game. This leads to a product being shipped only on basis of functional testing.

• The only savior in this scenario is once again automation. Testers need to ensure a regression back covering the critical path scenarios is automated with a good coverage (>60%) such that it can be triggered with the slightest change introduced.
• Also, it is advisable that automation suite is mature enough to run unattended so that regression can be carried out with minimal human intervention.

Conclusion: Working in agile way requires a major mindset shift, with regards to how planning, development and testing are handled. Also, skill changes (read automation) are needed to lead the agile projects to maturity. A very strong communication network is backbone of succeeding in any project but more importantly in Agile teams. Software Testers and other team members need to communicate clearly on estimations, what is done and what is pending. Process improvements needed should also be clearly called out in retrospective meetings so that they are taken care in next sprints. Gradually, the challenges should dissolve, and team should be able to mature in agile way of working.

*********

Articles and news about the software testing and quality assurance industry by premier portal for software testing jobs in India - Qualitician.