Notepad, check. Pen, check. Camera, check. Questions, check. Recorder, check! Oh wait! I have to check for a new update, just in case I am outdated.
She updated her app quickly and reached the coffee shop before time. She was excited, she was anxious. It’s an easy-peasy task, you just have to ask questions, record them and pour it out later– she kept repeating this in her mind.
But the moment Paula Hawkins walked in, she started feeling the breathlessness. Part excited, part nervous, she could feel her palms sweating. After exchanging some pleasantries they kicked off with the interview.
“Shall we start?”
“Yes, of course. Let me just..umm… Start this thing”
*she clicked on the record button and started talking, simultaneously taking notes.*
Unaware of what was happening after she clicked on that little red button, she went on with the interview, taking down notes as per her whims. Ignorant of the new updated feature — which now needed ‘long press and hold to start recording’, she finished the interview, smiling and pleased with herself. But as she looked down, in order to ‘STOP’ the recorder, she lost the fanatic feel of doing a good job!
A little hover over the button said “long press and hold to start recording” which was an obvious slip.
Game over. She probably lost her mind or her job. Or both. But that’s none of our concern. Right? She should have been more attentive.
Lyla is a fake person. She did not meet Paula Hawkins ever. But the incident is real. Only, it happened with an acquaintance and I figured out it wouldn’t be fair to name her. This incident forced me to shine some light on the importance of usability testing and why it’s more important to pay attention to user’s behavior while they interact with the product.
They say, never talk to your users, they hardly know what they want. But they missed out on the summary-
“To design the best UX, pay attention to what users do, not what they say. Self-reported claims are unreliable, as are user speculations about future behavior. Users do not know what they want.”
Usability testing- What is it?
Usability testing is like black-box testing of an application to ascertain if the product built is convenient to use and easy to learn.
These are methods of testing and observing the behavior of the users to find out what works and what doesn’t work. Users are given specific tasks to complete and when they are at work, observers watch their body language, facial expressions, emotions and encourage them to “think aloud” i.e. speak up whatever comes to their mind while using the product. Doing this exercise we can get qualitative and quantitative data and figure out usability issues with a product.
So, why usability testing is important?
Doing usability testing the right way, at right time with the right set of people reduces the risk of building the wrong product; thereby saving time, money and other precious resources. In other words, if done at an early stage when the product is at paper prototyping stage, it finds the problems when they are easy and cheap to fix. And when done on a product which has attained maturity it helps to understand user’s success rate and time spent to complete a task. There are hundreds and thousands of cases when usability testing proved to be a good exercise in terms of ROI.
For example, a slight tweak in design suggested by usability testing for Mac’s UI, the company got 90% fewer support calls.
Need more clarity about why usability testing is a good idea? Here you go-
#1. To check if product meet user’s expectations
#2. Matches business decisions to real-world use
#3. Removes flaws in the product
#4. Allows you to see how successful users are with their tasks
#5. Useful for getting user reactions and feedback about the product
What are the types of data that we can get as a result of our analysis?
Two types of data results received are — quantitative and qualitative. Usability testing is largely a qualitative research technique and is not driven by statistics like surveys where lots of people participate. Usability testing is done using a small set of people, usually five to seven.
Qualitative methods are very useful to test the stress response of the users like their body language, movement of hands, expressions on the face and squinting eyes especially doing a test on a mobile device.
The metrics we get after usability testing can be quantitative as well. For example, time spent on doing a task, success and failure rates and also the efforts, like how many clicks a user needs in order to complete a task.
Is there a need to record all the metrics obtained from a usability testing?
Yes, keeping a record of the metrics is very important. Why? Because usability testing is not just for designers to understand how to make better designs but it is also an important tool to influence the rest of the stakeholders like clients, their sales/support team, project managers, developers, other designers etc.
Every stakeholder involved may have a different point of view for a design decision. Being subjective by nature, design decision often leads to long debates among stakeholders. Most often design decisions are influenced by a person who holds the highest position among fellow stakeholder or has superior oratory skills.
In short, metrics help us in iterating and validating design concepts. It gives objectivity to design debates and it helps in taking fact-based design decisions.
At what phase of the design process usability testing is recommended?
When it comes to usability testing there are two terms often referred by big names of the UX industry (like Jacob Nielsen) and these terms are Summative Test and Formative Test.
These tests are done at different stages of the design process. They are as explained below:
Formative tests are low-fidelity tests (to gain quick insights)-
#1. During the very initial development phase using paper prototypes
#2. It can be done anywhere and a formal lab is not required
#3. It can be done just between a moderator and a participant
The results from a formative test may include-
#a. Users’ comments in the form of “Thinking Aloud” narrative i.e. their emotions, confusion sources and their reasons for actions.
Summative tests are high fidelity tests (to capture metrics)-
#1. These are carried out at the end of the development stage
#2. At this stage usability of a product is validated
#3. This gives an answer to the question “How usable the product is?”
#4. This gives a comparison against competitor products
#5. Conducted in usability labs or remotely using many tools available where users can do the test using their computers or mobile phones
The results from summative tests may include-
#a. User’s success rate to achieve a goal
#b. The time spent on completing a task
How many users are required to conduct the testing?
“Elaborate usability tests are a waste of resources. The best results come from testing no more than 5 users and running as many small tests you can afford”
Jacob Nielsen“It is widely assumed that 5 participants suffice for usability testing. In this study, 60 users were tested and random sets of 5 or more were sampled from the whole, to demonstrate the risks of using only 5 participants and the benefits of using more. Some of the randomly selected sets of 5 participants found 99% of the problems; other sets found only 55%. With 10 users, the lowest percentage of problems revealed by any one set was increased to 80%, and with 20 users, to 95%.”
Laura Faulkner
Who among them is right?
It depends on what type of test we are doing and where we are doing it. For example, if we are doing a low-fidelity formative testing we can do away with a small sample size. But if we are doing a summative testing we need a bigger sample size. In the type of testing where we are comparing our site to competitor’s website by using an online tool which is cheap and fast, we can use a large sample size. But we should keep in mind that these online tools like UserTesting or Loop11, they don’t capture metrics. It’s us who has to be aware of how all the participants did it.
So how to prepare a test plan?
That is certainly a good question. You have an inquisitive mind, I must say. But don’t you think that will be too much to digest in one bite?
Still, if you are really into it, give these things a thought-
-Decide what areas to concentrate on
-Determine potential usability issues
-Determine what tasks you want to test
