A test strategy is the guidelines to determine the road map to follow for the test types and cases defined in the testing plan. It is a holistic view to achieve highest possible quality at the end result of a project. A typical modern test strategy deals with –
- Test plans
- Test objectives,
- Testing approach
- Executing environment
- Automation test strategy and tools
- Prioritization and risk analysis
- Agile test strategy
From broader perspective test strategy can be viewed from three angles in a software test life cycle (STLC) –
- Test Devlopment
- Overall factors related to entire Test Strategy
Is the test strategy template final answer?
Every company follows their own test strategy template. But it is obviously not a good approach to blindly follow a standard. Every project and client has their own requirements. Moreover, every project may deal with different challenges. Hence, proper test strategy should act as a contingency plan. Let’s consider the two scenarios –
- Let say, you are going to test a website with rich UI and an external web service which is maintained by some external vendor. Here you are going to deal with two entirely different components. Hence the test strategies will be different. For UI part it could be a blend of manual and automation testing. Whereas for web service part you need to first check the overall functionality of the service and then you can set the automation process. Now, when it comes to system testing, scenarios must be determined based on customers’ perspective and not developers.
- Another common scenario is the mismatch of budget and time when dealing with the quality issue. Every testing project has its pre-defined fixed budget and timeline. But the timeline like start date can skip due to development issue, resource crunch or environment issues. How will you deal in such jinxed situation despite a pre-defined testing strategy? Will you compromise the budget or timeline or quality of testing? In each case, you are going to lose the game. Hence, better not to scapegoat yourself and define the proper strategy on the go to overcome the issue.
Modern test strategy mostly deals with prioritization and risk analysis and not just waterfall model of STLC.
Common test strategy approaches followed in Waterfall Model
Quality based strategy
In this strategy, stress is given on the right quality level of the products or services. Here all testing areas may not be covered. This means you can cut down testing in less important areas. But here one should consider the relationship between qualities with financial aspects of a business. Although focusing on quality can dramatically improve the technical performance of a product but on the other side, it may fail to address many fundamental issues that may impact a company’s long-term marketing and financial performance.
For example Y2K problem. Here, a product may fail to work on 29th February 2000 which is still acceptable. Although if it would fail to process the order after 1st Jan 2000 that was not acceptable.
Here testing is done for the most important functions and its critical in overall business. Once you find out the worst areas of development and design, you can find out more serious defects. When the possibility of potential business loss is identified through such strategy, management will be motivated to postpone the release. This will allow more testing time. The strategy is based on risk analysis where factors are scaled from 1 to 5 and the highest point is given to the worst factor.
The factors are defined based on the possible defect areas –
- Critical areas
- Visible areas
- Usage frequency
- Complex areas of the software/development
- Changed areas which can be defected generator in future
- Impacts due to use of new technology and methods
- Resource quality and numbers
- Time criticality
In each areas probability of failure is determined with calculation.
Cutting down the cost of automation
Although you can not 100% automate overall testing but automation testing is a good solution if you want to make testing cheaper. The possible risk lies here – if you are adopting automation first time and skilled resources.
Define the entry criteria for testing
To reduce the testing burden sometimes it is required to define few entry criteria of the product from the quality perspective. Entry criteria can include –
- Minimum reviews having been done before starting of testing
- Minimum level of test coverage performed in unit testing
- Certain level of product reliability
Test strategy followed in Agile Framework
Agile model demands stepwise releases. Hence the best strategy for an agile model is to test the most important functionalities of the application while delaying the others. This helps in identifying the defect-prone areas of a product. Moreover, agile model testing is performed in short sprints. Hence, strategies are focused on few requirements and user stories. Below diagram shows the test quadrants for agile testing where for each quadrant strategy must be made based on 5W1H criteria, i.e. WHY, WHO, WHAT, WHEN, WHERE and HOW. One of the popular agile testing frameworks used in the market is ATF.
Test strategy for DevOps
DevOps is an extension of Agile methodology which combines development and operations within the same cycle.In agile development it is continuous integration between development and QA operation, in DevOps, the code is deployed in the production after QA operation for each sprint. Hence, every action in this process needs to be fully automated to stress upon code quality. Same applies to the testing operation and test cases must be automated and achieve 100% code coverage.QA must assure an automated deployment. Hence a more matured automated testing framework is applied for DevOps scenario. Running all the regression test cases is not the strategy applicable for DevOps. Rather automated cross-platform and cross-browser testing for important scenarios are stressed upon. The overall process demands a close coordination between QA and development team. Prompt and proper reporting on the test results is essential to ensure overall quality delivery.
Automation Test strategy
One of the core testing methodology for today’s agile development is automation testing. Few thumb rules are followed in test automation like –
- Prevention rather than defect detection
- Quality over Quantity
- Quick feedback
- Frequent regression testing through automation which may involve Smoke regression pack, functional pack, E2E pack
- Automated Unit test
- Automated integration test
- Application test
The following are the building blocks of automation test strategy
- Selection of right automation tool for application – It is one of key to the success of any automation testing.
- Short listing of test cases – Through analysis of test plan is essential to shortlist the test cases which are most effective for automation.
- Setting up test environment – This is another building block to success. The rule of thumb is the test environment configuration must match with stage and production environment.
- Maximize the use of automation test scripts – This will lower the cost of testing
- Clear instruction on execution and management of test cases
- Focus on result – Outcome of each automation test case should be measurable and matrix-based to qualify the standard of the automation process.
- Use of automation maturity model – There are different maturity models used to revamp the test strategy like –
- Initial Phase
- Managed Phase
- Define Phase
- Measured Phase
- Optimize Phase
- Select the best automation framework for the project – Once the test cases are finalized for automation testing, the next important step is to select the right fit from the common automation testing framework. It could be any from the below list based on the project requirement –
- Data Driven Framework
- Keyword driven framework
- Modular based framework
- Hybrid framework
Mobile Testing Strategy
Mobile application testing needs more intensive attention as it runs on multiple OS, multiple platforms and on versatile mobile features like screen resolutions, screen sizes, OS versions, country locations and much more. To cover up all the criteria either mobile testing lab should be established otherwise testing in a cloud environment is the best idea. A cloud environment testing platform such as Crowd Source enables an application to be tested by numerous testers across the globe. Thus it covers all possible OS criteria, device features, locations and most importantly load testing similar to a real-time scenario.
Conclusion: Test strategy is not something that has to be strictly followed as documented. Rather it an overall test execution process and must be defined based on several criteria as discussed in the article. However, a proper test strategy plan and execution of that can definitely improve the testing process and quality of the software.