Call + 44 (0) 1 225 840 490
or email
Usability testing
Website usability testing is used to evaluate a web site by testing how real users use it and how that compares with how they are supposed to use it. In website testing the aim is to observe under controlled conditions the errors and possible areas of improvement. Usability testing can look at:
- Performance – how easy is it to do basic tasks such as creating a new account, and making a purchase?
- Accuracy – how many mistakes did people make and were they fatal or fixed?
- Recall – how much does the person remember afterwards or some time later?
- Emotional – how does the person feel about the site’s ease of use and do they get frustrated by anything?
By testing a website early and often, usability testing lets you identify and fix problems before they get coded and so the less expensive the whole development.
There are some issues with usability testing; for example you have to make sure that you are testing the site not the users and recognize the differences between objective (e.g. success rate, time taken) and subjective measures (e.g. user satisfaction). You also have to understand that objective and subjective measures do not always tally ‘logically’ with each other as time taken may be long at the same time that satisfaction is high.
So some principles that help are:
- Test behaviour and focus on what people actually do
- Test the critical factors
- Don't ask testers to do any redesign
Test behaviour and focus on what people actually do
Usability testing is not market research. Do not try to learn about brand values or how much people will pay – usability testing is about what people do and how they feel about the usability. If they are getting frustrated and want to throw the computer across the room because of the way the site works and you can find out why that’s helpful, but it’s not going to help you rebrand.
Test the critical factors
Make sure you are testing things that are critical to both the user and your organisation. What specific objectives do people have in mind when visiting a site (research? making a purchase?) as it’s those objectives you need to test your site against.
Don't ask testers to do any redesign
When participants struggle with some aspect of the site, it's very tempting to ask them how they would like it redesigned. However, generally participants don't know what's possible and asking them to redesign something tends to make them focus on the blindingly obvious because they are ‘under pressure’. You are asking them to identify what is wrong, not how to fix it.
So, how do you go about doing usability testing?
- Develop test plan
- Prepare materials
- Get participants and observers
- Conduct the sessions
Develop test plan
All you have to do here is decide:
- the test’s objective(s)
- the main questions
- how you will carry out the test
- how you will measure results
- the characteristics of those who will be invited to participate
Prepare test materials
The way you design tasks can have a dramatic outcome on results. Participants generally want to please you by following your directions. If the tasks direct participants to take a certain path, that's the way they'll go. If that is not what real users do in the true context of the design's use, then you may get incorrect results, so do not be too specific as to how but ask what you want them to do. e.g. “Please buy 6 items at random, remove one from the basket and complete the purchase.”
In the test materials, include any specific questions you might want to ask after the task(s).
Get participants and observers
Firstly get people who already know about what you’re testing and some that don’t – they are likely to provide different results.
Secondly, you only need 5 participants. Jakob Nielsen [http://www.useit.com/jakob/] has shown that the best results in usability testing come from testing no more than 5 users and running as many small tests as you can afford. This may seem counter intuitive. Surely a test with hundreds of users will get the best results?
Nielsen has shown statistically that zero users give zero insights (wow!) but as soon as you collect data from a single test user you will have already learned almost a third of all there is to know about the usability of the design. The second user adds some new insights, but not nearly as much as the first user did and so on. As you add more and more users, you learn less and less because you will keep seeing the same things again and again. There is no need to keep observing the same thing multiple times. After the fifth user, you are wasting your time by observing the same findings repeatedly but not learning much new.
Whilst he has found you need to test with at least 15 users to discover all the usability problems in a design he recommends testing with 5 users as it is better to do several small tests. The first study with 5 users will find 85% of the issues so you fix these problems in a redesign and test again. A second test will discover whether the fixes worked or and discover most of the remaining 15% of the usability problems. There will still be 2% of the original problems left but they come out in the third test.
Conduct the sessions
There are many ways to conduct testing. W. Craig Tomlin has a list of tools that you can use at http://www.usefulusability.com/24-usability-testing-tools/ if you are happy to do it yourself. Otherwise commission your agency to do it for you.
That’s it. Hope it’s helpful.
Written by Richard Hill
Newsletters
2012
2011 
2010
2009
2008
2007
2006
2005
Essays
