Software Testing - Usability Testing

Usability testing is the process of observing users’ reactions to a product and adjusting the design to suit their needs. Marketing knows usability testing as “focus groups” and while the two differ in intent many of the principles and processes are the same.

In usability testing a basic model or prototype of the product is put in front of evaluators who are representative of typical end-users. They are then set a number of standard tasks which they must complete using the product. Any difficulty or obstructions they encounter are then noted by a host or observers and design changes are made to the product to correct these. The process is then repeated with the new design to evaluate those changes.

There are some fairly important tenets of usability testing that must be understood :

•     Users are not testers, engineers or designers – you are not asking the users to make design decisions about the software. Users will not have a sufficiently broad technical knowledge to make decisions which are right for everyone. However, by seeking their opinion the development team can select the best of several solutions.

•     You are testing the product and not the users – all too often developers believe that it's a 'user' problem when there is trouble with an interface or design element. Users should be able to 'learn' how to use the software if they are taught properly! Maybe if the software is designed properly, they won't have to learn it at all ?

•     Selection of end-user evaluators is critical –You must select evaluators who are directly representative of your end-users. Don't pick just anyone off the street, don't use management and don't use technical people unless they are your target audience.

•     Usability testing is a design tool – Usability testing should be conducted early in the life-cycle when it is easy to implement changes that are suggested by the testing. Leaving it till later will mean changes will be difficult to implement.

One misconception about usability studies is that a large number of evaluators is required to undertake a study. Research has shown that no more than four or five evaluators might be required. Beyond that number the amount of new information discovered diminishes rapidly and each extra evaluator offers little or nothing new.

And five is often convincing enough.

If all five evaluators have the same problem with the software, is it likely the problem lies with them or with the software ? With one or two evaluators it could be put down to personal quirks. With five it is beyond a shadow of a doubt.

The proper way to select evaluators is to profile a typical end-user and then solicit the services of individuals who closely fit that profile. A profile should consist of factors such as age, experience, gender, education, prior training and technical expertise.

I love watching developers who take part as observers in usability studies. As a former developer myself I know the hubris that goes along with designing software. In the throes of creation it is difficult for you to conceive that someone else, let alone a user (!), could offer better input to the design process than your highly paid, highly educated self.

Typically developers sit through the performance of the first evaluator and quietly snigger to themselves, attributing the issues to ‘finger trouble’ or user ineptitude. After the second evaluator finds the same problems the comments become less frequent and when the third user stumbles in the same position they go quiet.

By the fourth user they’ve got a very worried look on their faces and during the fifth pass they’re scratching at the glass trying to get into to talk to the user to “find out how to fix the problem”.