Software Testing - Non functional testing

Testing the design

Requirements, design and technical specifications can be tested in their own right.

The purpose of evaluating a specification is threefold:

•     To make sure it is accurate, clear and internally consistent (verification)

•     Evaluating how well it reflects reality and what the end-user expects (validation)

•     Making sure it is consistent with all previous and subsequent phases of the project

The technical specification is an embodiment of the requirements which should then flow through to subsequent phases like production and testing. If the requirements are poorly specified then not only will the product be inadequate but will also be incredibly difficult to verify.

If the technical specification is out of synch with the requirements then it is likely that the development team will be well on its way to producing the wrong product. Because these documents are often produce in parallel (i.e. the steps in the waterfall model overlap) it is very common for discrepancies to creep in.

Each assertion in a specification should be reviewed against a list of desirable attributes:

•     Specific – it is critical to eliminate as much uncertainty as possible, as early as possible. Words like "probably", "maybe" or"might" indicate indecision on the part of the author and hence ambiguity. Requirements including these words should be either eliminated or re-written to provide some kind of surety as to the desired outcome.

•     Measurable – a requirement which uses comparative words like “better” or “faster” must specify a quantitative or qualitative improvement must do so with a specific value (100% faster or 20% more accurate).

•     Testable – in line with the above, a requirement should be specified with some idea of how it should be evaluated. A requirement which is not testable is ultimately not ‘provable’ and cannot therefore be confirmed either positively or negatively.

•     Consistent- if one requirement contradicts another, the contradiction must be resolved. Often splitting a requirement into component parts helps uncover inconsistent assumptions in each, which can then be clarified.

•     Clear - requirements should be simple, clear and concise. Requirements composed of long-winded sentences or of sentences with multiple clauses imply multiple possible

outcomes and so lack clarity. Split them up into single statements.

•     Exclusive – specs should not only state what will be done, but explicitly what will not be done. Leaving something unspecified leads to assumptions.

Further, it is important to differentiate requirements from design documents. Requirements should not talk about “how” to do something and a design specs should not talk about “why” to do things.