As an innovation studio, user testing runs in our veins. 

And we don’t mean elaborate and costly user testing of a finish product, but a pragmatic prototyping approach that starts from day one. 

In this article we’ll run through frequently asked questions, bust some myths and finish with a step-by-step guide to getting started.

Article sections:

1. What is user testing and why is it crucial for product teams’ success?
2. Obstacles for product teams & how to tackle them
3. User testing aims: your product might be usable, but is it desirable?
4. User testing types: in-person or remote?
5. Let’s get started: how to prepare for user testing
6. How to run the perfect user test
7. How to use data for maximum results

What is user testing and why is it crucial for product teams’ success?

User testing is a broad field, so let’s start by defining what type is most relevant and useful for product teams.  

As Design Sprint experts and advocates, we preach user testing from day one. So a lot of what we’ll reference in this article is based on early stage user testing. This means putting, not a fully finished product, but a simplified high fidelity prototype into the hands of real users, and exploring and documenting their reactions and opinions.  

If you’re familiar with Design Sprints, this would be day 5 of the process, but the same principles can be applied to user testing at any point in the development cycle. 

In terms of format, this is a pragmatic and open exploration, rather than scientific tests like eyetracking or AB testing. You give the user access to the prototype, and with some loose guidance and thoughtful questioning, you observe verbal and emotional reactions, as well as obstacles and ‘pain points’. 

This type of testing is crucial because, by testing a product before diving into coding, you cut the biggest risk facing any product team (that the end user will just not ‘get’ the product), and therefore save countless hours, dollars and leadership brownie points in the long term. 

Sounds great right? So why aren’t we all doing it?

{{banner-yellow="/banners"}}

Obstacles for product teams & how to tackle them

Pretty much all of the objections we hear when it comes to user testing fall into three categories:

Money
Time
Fear

And we get it. There is an investment involved. It is inherently scary to put your hard work under the microscope. 

But here’s how to cut both risk and nerves: 

Run tests lean

Within Design Thinking principles, you can effectively test your idea with as few as 5 real users. And if you’re wondering it this works for large markets, remember this is methodology pioneered at companies with user bases in the billions (Google, Netflix, Spotify… et al).

Since tests can be run remotely (find out how below), you will only take a couple of hours of their time, and there are plenty of creative and hybrid ways to compensate. 

Run tests early

If you don’t do any testing til the final product is coded, polished and ready, then of course there’s a massive fear factor. 

This is why prototyping and validating needs to start from day one. 

Yes, the prospect of hearing an ugly truth about your work is scary, but wouldn’t you rather hear it before you’re months over deadline and thousands over budget? 

Testing is all about detecting problems before they’ve done damage.

To your timeline, your budget or your reputation. 

{{banner-green="/banners"}}

Run tests often

Here’s the key – making user testing a standard part of every stage of the development cycle. 

Each time you repeat the process:

• It’ll take less time 
• Your team will get more efficient at preparing and running tests
• You’ll know how to squeeze more juice out of the results
• You’ll get quicker at detecting issues, saving time and money on pursuing dead ends

A win-win cycle. 

So now we know the why of user testing, let’s get into the details:

User testing aims: your product might be usable, but is it desirable?

The most common mistake we see during user testing is when teams equate usability with value. 

The waste bin of innovation is overflowing with products that were perfectly usable but ultimately weren’t valuable to users. (Just ask Google about their Glass project.) 

When assessing what type of features to include in the prototype and the direction of interview questioning, it’s important to be clear that we’re not only looking for basic UX interactions, but the way the product makes the user feel. 

Is it valuable enough, impactful enough, desirable enough that they would click pay?

As always, continuous interdepartmental collaboration is key. Involving the perspectives of designers and engineers, as well as marketing and customer service, when planning questions, defining the prototype and when analysing results will 10x productivity and insights. 

User testing types: in-person or remote?

In-person user testing is still a great option for qualitative insights. Being in the room allows for a more instant connection with the participant and a more direct experience of their behavior and body language.

However, as hybrid and remote working becomes ever more common, people are increasingly comfortable with video conferencing. There are multiple software options that allow for detailed real time observation. You can do this through a basic platform like Google Meets, Zoom or Teams and/or a more specialized software like Figma, which allow an even more interactive and adaptable experience. 

If done well, remote interviews can produce just as valuable results as the in-person format, with the added benefits of lower costs, time investment and logistical complexity, as well as a wider selection of participants. 

Let’s get started: how to prepare for user testing

First, it’s important to note that experience and training are crucial to running a powerful user test. The pointers below give you the basics of the process. We hope they’ll help you with improving user testing procedures, or give you some context on our approach as an external agency when we do this as part of a Design Sprint. 

Setting goals

Get clear on the fundamental aim of testing from the very start. 

As part of the alignment stage of a Design Sprint, we always discuss:

• The 2-year goal – where do we want to be with the product in two years
• The 3 “can we questions” that we need to answer in order to achieve that goal

Those three “can we questions” form the base of effective user testing, because our aim is to validate the current course of action and gain insights to create a roadmap to the 2-year goal. 

From there we design a static screen solution that we propose to answer these questions, leading to a clickable prototype, ready to test with our users.

Finding users

Product teams often get stuck on this point, but there are lots of creative ways to search for the perfect participants. 

For a wide market you can use forums, online groups or Facebook ads. For a specialist product or service, an existing email list, client database or professional network can be a better option. 

Keep next steps as streamlined as possible by sending applicants straight to a simple screener questionnaire, then automizing the process of sending the NDA and scheduling tool to chosen candidates. 

Incentives also depend on the brand, from discounts / free gifts to a simple Amazon voucher. 

{{banner-purple="/banners"}} 

Planning

While you’ll want to establish a natural, relaxed conversation, it’s important to outline how you want the interview to flow. 

Think about your aims and your “can-we questions”, as well as how long each section of the interview might take. 

The Five-Act Interview works well as a broad structure.

1. A friendly welcome to give the participant an overview of what to expect
2. General, open-ended context questions about the customer as a ‘warm up’
3. Introduction to the prototype – if remote, this is the time to share your link and ask the participant to screenshare 
4. Detailed tasks to get the customer reacting to the prototype
5. A quick debrief to capture the customer’s overarching thoughts and impressions. If you haven’t already, make sure you validate the can-we-questions here

Finally, before the test, make sure you’ve planned and trialled all of your tech, and informed the participant of any tech requirements at their end.

How to run the perfect user test

You’re all set for a productive interview. Here’s what to consider once you’re in the room:

Create the right ambience

Imagine yourself in the partipant’s chair and consider what will help them feel comfortable. Create a warm environment, and start with an informal chat to put them at ease. Above all, let them know what to expect.

Having a partner interviewer is great because it allows one of you to be completely focused on rapport and active listening, while the other can deal with any note taking, technical tasks, etc.

Ask the right kind of questions

The language you choose is key to squeezing the most juice out of an interview. 

Questions should always open ended. So avoid anything that could have a yes or no answer, and don’t give prompts or options. 

Consider the difference between…

Do you like feature X? 
Do you like feature X, or would you prefer it to be more like feature Y?’

vs 

What do you think of feature X?

A handy cheatsheet:

Open ended questions, usually start with:
"Who...?" "What...?" "Where..?" "When...?" "Why...? "How..?"

Avoid questions that start with:
"Would...?", "Do/Does..?", "Is it ...?", “Are you…?”

It goes without saying that leading questions – eg a statement followed by "…right?’",  "…isn’t it?" or similar  - are a big no-no.

Dig deeper

While planning is essential, staying curious and open is fundamental to being a great interviewer. Even if you think you’ve got the full answer, ask another why. The biggest lightbulb moments often come from questions that weren’t in your plan.

{{banner-blue="/banners"}}

How to use data for maximum results

Once you’ve done your test is when the work really begins. Of course, this could be a whole blog article in itself, but here’s how we usually get started:

All the team members involved should watch the recording and extract their own insights, each on a sticky note.

You’ll be amazed how many different points are picked up by diverse perspectives.

Then we bring them all together on one big board and group them into:

• Praise
• Critique
• Idea
• Highlight
• Easy fix

From there, similar to every phase of a Design Sprint, we run through various exercises to prioritize and align on the most significant points to take into the iteration phase.

Read more: How Design Sprint helps enterprises cut costs, time and risk

So that rounds up our User Testing 101 overview. We hope you’ll have a running start with our tips on the benefits of user testing, how to plan & run a great test, and what to do next. 

If you’d like to learn more about how we can teach your team to become expert testers, or how we integrate user testing into our Design Sprint, book a free consultation with our team of experts.
‍

{{banner-newsletter="/banners"}}