Usability testing is like black-box testing of an application to ascertain if the product built is convenient to use and easy to learn.
These are methods of testing and observing the behavior of the users to find out what works and what doesn’t work. Users are given specific tasks to complete and when they are at work, observers watch their body language, facial expressions, emotions and encourage them to “think aloud” i.e. speak up whatever comes to their mind while using the product. By doing this exercise we can get qualitative and quantitative data and figure out usability issues with a product.
So, why usability testing is important?
Doing usability testing the right way, at right time with the right set of people reduces the risk of building the wrong product; thereby saving time, money, and other precious resources. In other words, if done at an early stage when the product is at the paper prototyping stage, it finds the problems when they are easy and cheap to fix. And when done on a product that has attained maturity it helps to understand the user’s success rate and time spent to complete a task. There are hundreds and thousands of cases when usability testing proved to be a good exercise in terms of ROI.
For example, a slight tweak in design suggested by usability testing for Mac’s UI, the company got 90% fewer support calls.
Need more clarity about why usability testing is a good idea? Here you go-
#1. To check if the product meets user’s expectations
#2. Matches business decisions to real-world use
#3. Removes flaws in the product
#4. Allows you to see how successful users are with their tasks
#5. Useful for getting user reactions and feedback about the product
What are the types of data that we can get as a result of our analysis?
Two types of data results received are — quantitative and qualitative. Usability testing is largely a qualitative research technique and is not driven by statistics like surveys where lots of people participate. Usability testing is done using a small set of people, usually five to seven.
Qualitative methods are very useful to test the stress response of the users like their body language, movement of hands, expressions on the face, and squinting eyes especially doing a test on a mobile device.
The metrics we get after usability testing can be quantitative as well. For example, time spent on doing a task, success and failure rates, and also the efforts, like how many clicks a user needs in order to complete a task.
Is there a need to record all the metrics obtained from usability testing?
Yes, keeping a record of the metrics is very important. Why? Because usability testing is not just for designers to understand how to make better designs but it is also an important tool to influence the rest of the stakeholders like clients, their sales/support team, project managers, developers, other designers, etc.
Every stakeholder involved may have a different point of view for a design decision. Being subjective by nature, design decision often leads to long debates among stakeholders. Most often design decisions are influenced by a person who holds the highest position among fellow stakeholders or has superior oratory skills.
In short, metrics help us in iterating and validating design concepts. It gives objectivity to design debates and it helps in taking fact-based design decisions.
At what phase of the design process usability testing is recommended?
When it comes to usability testing there are two terms often referred to by big names of the UX industry (like Jacob Nielsen) and these terms are Summative Test and Formative Test.
These tests are done at different stages of the design process. They are as explained below:
Formative tests are low-fidelity tests (to gain quick insights)-
#1. During the very initial development phase using paper prototypes
#2. It can be done anywhere and a formal lab is not required
#3. It can be done just between a moderator and a participant
The results from a formative test may include-
#a. Users’ comments in the form of “Thinking Aloud” narrative i.e. their emotions, confusion sources and their reasons for actions.
Summative tests are high fidelity tests (to capture metrics)-
#1. These are carried out at the end of the development stage
#2. At this stage usability of a product is validated
#3. This gives an answer to the question “How usable the product is?”
#4. This gives a comparison against competitor products
#5. Conducted in usability labs or remotely using many tools available where users can do the test using their computers or mobile phones
The results from summative tests may include-
#a. User’s success rate to achieve a goal
#b. The time spent on completing a task
How many users are required to conduct the testing?
“Elaborate usability tests are a waste of resources. The best results come from testing no more than 5 users and running as many small tests you can afford”
Jacob Nielsen“It is widely assumed that 5 participants suffice for usability testing. In this study, 60 users were tested and random sets of 5 or more were sampled from the whole, to demonstrate the risks of using only 5 participants and the benefits of using more. Some of the randomly selected sets of 5 participants found 99% of the problems; other sets found only 55%. With 10 users, the lowest percentage of problems revealed by any one set was increased to 80%, and with 20 users, to 95%.”
Laura Faulkner
Who among them is right?
It depends on what type of test we are doing and where we are doing it. For example, if we are doing low-fidelity formative testing we can do away with a small sample size. But if we are doing summative testing we need a bigger sample size. In the type of testing where we are comparing our site to a competitor’s website by using an online tool that is cheap and fast, we can use a large sample size. But we should keep in mind that these online tools like UserTesting or Loop11, don’t capture metrics. It’s us who has to be aware of how all the participants did it.
So how to prepare a test plan?
That is certainly a good question. You have an inquisitive mind, I must say. But don’t you think that will be too much to digest in one bite?
Still, if you are really into it, give these things a thought-
-Decide what areas to concentrate on
-Determine potential usability issues
-Determine what tasks you want to test
