Software Testing - Test Preparation

Test Scripting

 

There are several schools of thought to test scripting.

 

In risk averse industries such as defence and finance there is a tendency to emphasise scripting tests before they are executed. These industries are more concerned with the potential loss from a software defect than the potential gain from introducing a new piece of software. As a consequence there is a heavy emphasis on verifiable test preparation (although observance of this verification might only be lip-service!) And in some industries, external compliance issues (legal compliance for example) mean that a script-heavy approach is mandated.

 

On the other hand, in Commercial-Off-The-Shelf (COTS) software development, a looser approach is normally taken. Since speed-to-market is more important than the risk of a single software defect, there is considerable latitude in the test approach. Specific test cases may not be documented or loosely documented and testers will be given a great deal of freedom in how they perform their testing.

 

The ultimate extension of this is exploratory or unscripted testing.

 

In this form of testing, there is a considerable amount of preparation done but test cases are not pre-scripted. The tester uses their experience and a structured method to 'explore' the software and uncover defects. They are free to pursue areas which they think are more risky than others.

 

Scripting, it is argued, is a waste of time. On a big project the amount of time spent on scripting can actually exceed the amount of time in execution. If you have an experienced, educated tester with the right set of tools and the right mindset, it would be more effective and more cost efficient to let them get at the software right away and find some defects.

 

 

 

There is also an important legal aspect to this as Cem Kaner points out in his book “Testing Computer Software”. If you are responsible for releasing a piece of software that causes financial loss you could be liable for damages. Further, if you cannot prove that you have conducted due diligence through adequate testing you may be guilty of professional negligence. One of the goals of test preparation therefore is to provide an audit trail which shows the efforts you have made to verify the correct behaviour of the software.

 

I sit somewhere in the middle of this debate.

 

Preparation is essential. Some scripting is good but too much is not. Exploratory testing relies on good testers and fails without them. But a lot of time can be 'wasted' in scripting methodologies by writing scripts that will never find a defect.

 

I do think that 'exploratory testing' produces better (thinking) testers than script heavy methodologies. In script heavy methodologies there is a tendency to believe the hard work is over when the scripting is done. A monkey could then execute the scripts and find defects – this is a dubious conclusion at best.

 

But sometimes all you have are monkeys and it is a more efficient use of resources to allow your experienced testers to script, and use the simians to execute.

 

In the end, don't let adherence to a particular methodology blind you to the possibilities of other approaches. Train your testers in all the possibilities and let them use their judgement.