How to Win a Software Testing Campaign the Sun Tzu Way

Usually, hundreds of happy path test scripts are created, each equally valued by the business users, without any consideration of their importance - little to no risk analysis or traditional negative or exploratory tests. We discover, during the test execution stage, that not all test scripts created are executed and all the system defects that are discovered come late in the testing schedule (very often after the delivery of the product). The net result is that you end with poor quality or are left wondering why QA efforts have returned low value. The defects reported, late in the testing cycle, catch the entire team unprepared to resolve the issues and there aren't enough resources to deal with the defects and lack of time. Does this story sound familiar? Are you encountering this behaviour-based approach on a daily basis? Do you find yourself asking for more help, and if you are engaging vendors, aren't they always there to help you and take your money? Keep Reading!

Our objective as test professionals is to discover system or application issues as quickly as possible so that we allow sufficient time for resolution. We fail our mission if we report the majority of system defects during the tail end of testing, thereby repeating the pattern above. We can break this pattern by having a well thought out strategy in place to deal with the situation at hand. We can turn to Sun Tzu and apply a lesson learned ages ago in his work "The Art of War".

"To secure ourselves against defeat lies in our own hands."
Sun Tzu.

We look at Sun Tzu's work as application and system testing is like warfare. We are required to coordinate resources to meet our testing objective.

1. Software testing, like warfare, depends on people and tools, as well as dealing with the effective use of time and resources.
2. Software testing is based on some assumptions, just like warfare.
3. In both, parallel activities are happening. For example, in war, a number of different battles take place concurrently; and, similarly, in testing, several types of testing and execution of different test cases or scenarios occur simultaneously.
4. Leadership, the environment, communication skills, and the support of all individuals involved, be they privates or testers, have a big impact on success.
5. Doctrines of careful preparation have become fundamental approaches to military and software testing.

In this article, we will map a few key strategies from the book The Art of War by Sun Tzu and how they relate to software testing.

We will try to apply Sun Tzu's ideas to the world of software testing in a way that is readable, useful, and practical. Using the principles of Sun Tzu, we will be able to avoid failure and meet our testing objectives. It is important to discover system or application issues as quickly as possible so that we allow sufficient time for resolution.

This article is intended to help you as a testing professional to open the door to sound test strategy and planning, which we believe are the keys to success.

According to Lionel Giles, whose translation of The Art of War which is freely available on the web, Sun Tzu said:

1) "By method and discipline are to be understood the marshaling of the army in its proper subdivisions, the graduations of rank among the officers, the maintenance of roads by which supplies may reach the army, and the control of military expenditure."

• In software testing, a good organization of the team, a clear understanding of tasks, well-maintained test cases and scripts (i.e., the roads analogy above), and control of testing expenditure are vital for successful project delivery.

2) "According as circumstances are favorable, one should modify one's plans."

• In software testing, think of writing and creating a test plan as a plan of actions. Planning without action is a waste of time. If you act without planning, you can end up anywhere. While test strategies remain constant during the project, tactics must be adapted to each new situation. Being successful in testing or in war requires simultaneous planning and action. The initial test-planning phase is important, but too much planning can also be disastrous (agile development, taken directly from Sun Tzu's book). Any test plan must be designed to allow for easy implementation of changes.

3) "All warfare is based on deception" - interesting and true.

• In software testing, be cautious with test automation, a common trap of automated test scripts that run for hours without doing anything. The key to success is to start with a limited number of automated test scripts that are well designed, need to be run often, and have easy maintenance. Look at your ROI and it will guide you to the next step and to the final victory.

4) "Attack when opposition is unprepared, appear where you are not expected."

• In software testing, you should start testing in the most vulnerable areas of the system or application in order to uncover showstoppers and defects that must be fixed first. Plan for exploratory testing.

5) "Military devices, leading to victory, must not be divulged beforehand."

• In software testing, as a tester, you should have, and always be trained to use, alternative testing tools and methods that are prepared in advance to use as contingency.

As you can see from the examples above, war and testing have a lot in common when it comes to planning and strategy. In war, let your objective be victory, not lengthy campaigns, and the same applies to software testing.

Here are a few concepts to consider when test planning, in order to save time:
• Improve communication between developers and testers.
• Reduce the number of cycles for fixing defects.
• Thoroughly describe the defect and the steps necessary to reproduce it.
• Use different automated tools where it is possible (this is usually highly effective).
• Have a contingency plan with all available alternative testing methods.
• Improve time management and leadership practices.

Let's spend a few moments on time management and leadership. When we refer to time management, we are talking about testing early. Find critical issues quickly so your development teams have a chance to resolve these issues, hopefully with permanent solutions. Rapid decision-making produces rapid test execution, so review and streamline as many decision-making points as possible in your project.

In general, software testing leaders must have profound knowledge of testing and theory as well as hands-on testing experience. Testing tactics begin with execution and are modified during the testing period. Methodology is only theory and it is experience that will allow you to solve problems in difficult testing situations. We believe this is a key ingredient for success, which is nicely expressed by these three gentlemen:

• "When I give a minister an order, I leave it to him to find the means to carry it out." - Napoleon Bonaparte

• "What you cannot enforce, do not command." - Sophocles

• "Don't tell people how to do things, tell them what to do and let them surprise you with their results." - George S. Patton

We hope this article has provided you with some basic fundamentals to test planning. As a recap:
1) After discovering the weak areas of your development environment, begin by testing the most critical areas of the system and finding the key issues quickly.
2) Target vulnerable segments in the system or application and test (attack) there first.
3) Define test scenarios around key customer functionality that evaluates system algorithms and then tests them.
4) Test at the boundaries between different systems or applications.