DevOps Testing Strategy

Overview: DevOps literally is a combination of two words- Development and Operations. It is basically a software development methodology that integrates all the Software Development functions from development to operations within the same cycle.

DevOps needs a lot of coordination among various stakeholders in the entire software development process. There is only one goal - build better, faster, and more responsive software. It is a cultural shift to remove the barriers between Dev and Ops to meet the need for shorter and more frequent software deliveries. One of the best practices for DevOps teams and agile methodology is to implement is CI/CD- i.e Continuous Integration and Continuous Delivery.

Continuous Integration and Continuous Delivery:

CI or Continuous Integration is a practice where developers frequently merge code changes into a central repository where automated builds and tests run. The changes are validated by creating a build and running automated tests against the build. This practice is usually adopted to avoid integration issues that may arise if we go for big bang integration mode.

As it is obvious, due to continuous integration, there is a lot of focus on testing to ensure that new code changes did not break the application. This is usually achieved by automation tests which automatically run every time the build is completed.

CD or Continuous Delivery is the extension to continuous integration process. It involves automatically deploying all code changes to a testing and/or production environment after the build is validated. Hence, we are building, testing, and deploying continuously.

Continuous delivery can also be extended to make it Continuous Deployment. With Continuous Deployment, every change that is built and tested successfully is released to customers without any manual intervention except in case of failures. Hence, continuous deployment is making the release pipeline automatic.

Traditional Testing Vs Testing in Agile Vs Testing in DevOps:

• Traditional Testing or we can say testing in Waterfall model used to start after the development was completed and was usually a manual process. Testing team was usually isolated and bug fixing was costly as it was very late in software life cycle.
• Agile model shifted testing to the left, against than the test-at-the-end approach of traditional development. In agile testing, code is developed and tested in small increments of functionality. Frequent interaction between an IT department and business users is required. Regular builds are delivered to meet the client requirement.
• Testing in DevOps is however quite different.
  •  DevOps focuses on constant testing and delivery while the Agile process focuses on supporting constant changes.
  •  DevOps brings development and operations teams together whereas Agile is an iterative approach that focuses on quick customer feedback and small rapid releases.
  • Agile does not calls our automation as necessary (although the success of agile requires automation) whereas automation is primary goal of DevOps.

In short, we can say that DevOps is Agile model which extends the cross-functional Agile team made up of software designers, testers and developers to include support people from Operations.

DevOps Testing Strategy:

Software testing is done continuously at each stage in the DevOps model even after applications are in production. The main aim of DevOps model is to automate as many as manual processes to make things efficient and deliver in quicker cycles. Automation is a key ingredient for a successful DevOps lifecycle, particularly test automation. It is important because it makes instant feedback loops possible. The code once developed can be tested in a faster and efficient way and defects detected and fixes provided. In practice, continuous testing can only be achieved by automating tests.

Automation also reduces the risk of error that typically occurs when the tests are repeated like regression. It also helps to enhance the scope of testing with lesser or equal number of resources, lower risk and higher speed, hence enabling continuous testing which is the core of DevOps.

However, automation is easier said than done. The strategy on how to automate is very important to implement a successful DevOps model. Some of the best practices with regards on how to start test automation for DevOps implementation are -

• First chalk out the release pipeline
  • Identify all stages of release - Example - Dev, QA, Stage, Prod
  • Identify necessary requirements in a build’s journey to production.
  • Identify feedback mechanism required for quick error detection and resolution.
  • Put together all SOPs involved with a release cycle.

• Find the right automation tool
  • A DevOps automation tool should be usable by Developers, Testers and Operations alike.
  • It should integrate seamlessly into your CI/CD pipeline and or with CI/CD tools (like Jenkins)
  • It should be independent of underlying infrastructure and should be easily maintainable.

• Build automation flows gradually
  • Start your automation journey with flows which are easy to automate and are repetitive.
  • Gradually build your test suite to include more flows to increase coverage

• Keep the tests segregated
  • Do not bundle multiple validations in a single test. It helps to isolate failures.
  • Build reusable components

• Ensure collective ownership
  • Adopt a test automation platform that all testers can work with, so that automation becomes a natural part of the daily work of all team members.
  • Bring a cultural shift that quality is everyone’s responsibility.

With all said about automation, we should be careful not to automate the flows which do not add value to the process. DevOps does not mean that everything must be automated regardless.

Also, while automating tests, we must adhere to some best practices like,

• Using good test data
• Running smaller test suites in parallel to increase speed.
• Keep updating the test suite to ensure that suite can actively detect bugs
• Use metrics to evaluate the success or failure of testing. Metrics enable teams to foresee if the number of bugs will increase or decrease with changes introduced to the system over time.

Conclusion: The success of testing greatly depends on the best practices adopted in your DevOps setup. Moreover, a cultural shift is also required to make DevOps a success. Automation and its related tools are at the heart of DevOps testing strategies. DevOps is an ideal solution to become competitive and adapt to your customer base.