User Research summary

We live in a data-driven world, and that’s good news for businesses. Companies have never had so much information about customers at their fingertips. But all that data can be overwhelming. If you’re not asking the right questions, you’ll never discover what your customers truly want and need. 

That’s why it pays to think carefully about your user research methodology. With so many different approaches out there, you want to make sure you’re picking the right method for your business. Luckily, the insights of seasoned professional Stephanie Marsh make this easy to do.

Marsh is a user experience and research consultant who has worked with everyone from the British government to EasyJet. As actionable as they are insightful, these blinks offer a crash course in making your research count.  

Along the way, you’ll learn

• how to use neutral questions;
• how to tackle sensitive topics; and 
• why observation is just as important as the questions you ask. 

The data generated by user research is only valuable if your participants are honest. But getting an honest answer is often harder than you may think. Sometimes people are just too polite to tell you they have difficulty using your product. So how do you get the information you need?

For starters, don’t just focus on what they’re saying. Pay attention to how they behave. Consider this example from the author’s experience as a consultant. When she was working with the British government, she assessed how user-friendly its website was. To do that, she ran a user test in which participants completed a series of tasks on the site. Some tasks were simple, while others were more complicated. 

One participant proved harder to deal with than others. Overall, she seemed to have had a fairly smooth user experience. But when the author quizzed her after the trial, she claimed that the site was awful. She went on to criticize the current prime minister and his political positions. It was clear that her feelings about the government influenced her perception of the website. That’s a great example of how external factors can sway opinions. It also shows that it pays to observe behavior, rather than just ask questions! 

Another way of getting usable data out of your research is to formulate your questions as carefully as possible. Neutrality is the watchword here. When you ask a question like “Do you like this?” you may put participants on the spot, and they may feel compelled to give a positive or negative answer. That often distorts their actual experience, which might have been less clear-cut. 

Questions like “What, if anything, did you like or dislike about this?” are much more balanced. They give participants the chance to comment on positive and negative features without either endorsing or rejecting your product entirely. 

Now you know that user research is vital to determining the viability of your product. But how do you actually do it? It’s time to talk about methodology. Before you decide on your approach, you need to know what kinds of methods are out there. 

Let’s kick things off with moderated usability testing. It’s a mouthful, but it’s a pretty simple concept. “To moderate” in this case means to monitor. In this approach, you observe a participant try out a product or service. Sometimes you’ll literally look over their shoulder. Other times, you’ll use screen-sharing tools to watch what they’re doing remotely.

Moderated tests have one big advantage – they’re highly interactive. When participants get stuck or run into problems, they can ask you questions directly. That gives you an insight into their experience and tells you what you’ll need to fix before rolling out your product. You can also ask follow-up questions and have deeper conversations with trial users.

Finally, conducting moderated tests is relatively cheap. If you’re in the room, all you need is a pen and paper or a computer to jot down your observations. 

That said, moderated usability testing isn’t always the right option. While it might not cost much financially, it’s pretty time-intensive, as you’ll have to be present at each session. Then there’s the logistical headache of agreeing on a place and time with participants.

This issue means that you can only use this method with a small number of people. If an average interview takes 45 minutes, you’ll only be able to squeeze so many participants into a day. While your research will lead to plenty of insights, it won’t be statistically significant. 

That matters because you need to convince your superiors that your findings are worthwhile. Lots of bosses just won’t be impressed with data gathered from five people, even if you explain the value of this in-depth research. In other words, moderated usability testing can be a tough sell.

You’ve probably already guessed one of the obvious alternatives to moderated usability testing – that’s right, unmoderated usability testing. The set-up is almost identical to the one we discussed in the previous blink. However, there’s one important difference – participants are left to their own devices as they complete tasks and test your product or service. That makes your life pretty easy, right? 

Well, yes – unmoderated usability testing definitely has its advantages. First off, you can carry out this kind of testing remotely, allowing participants to take part wherever and whenever suits them. That’s handy if you’re attempting to create a large data pool and don’t have time to schedule individual meetings with each user. One added bonus is that you can reach out to people who live in hard-to-reach locations, as well as people who don’t have the financial means to travel to you. 

Unmoderated usability is also fast. A large-scale study can usually be completed in just a couple of days. If you’re pressed for time and need a quick turnaround, this is the method for you. 

There are drawbacks, however. If you’re asking participants to complete a task without assistance, you might end up excluding certain demographics. A remote computer test, for example, won’t be accessible to people without an internet connection. Elderly participants will probably struggle with computer-based tasks. That’s only important if you intend to find customers within these groups, of course, but it is worth bearing in mind. 

The other issue with unmoderated usability testing is the quality of the data. While you’re sure to end up with lots of information, it will inevitably be fairly superficial. Some of it won’t be usable at all – that’s the downside of not being onsite to help participants with their tasks. In fact, you won’t have much control over the entire process at all. 

So far we’ve looked at a couple of research methods which might be new to you. In this blink, we’ll turn our attention to one you’ve definitely heard of before – the survey. It’s probably the most widely used and well-known methodology in user research.

Let’s start with the obvious stuff. A survey is the statistical equivalent of an industrial trawler – it collects vast amounts of quantitative data. The key here is a representative sample – a group of people that represents your target market as a whole. As a rule of thumb, this group should contain at least 2,000 individuals. Any less than that and your data won’t be statistically significant. 

So, say 100,000 people use your services or buy your product every year. If you select 2,000 of these users, you can be reasonably confident that your data will represent the experiences of all your customers. 

That brings us to survey design. To craft an effective survey, pay attention to a couple of factors. First off, you should have a clearly identified purpose. Secondly, you’ll need to target a pre-selected group of participants. Finally, you’ll have to pick a medium – are you going to use web-based tools, for example, or stick to an old-school paper-and-pen approach? 

It’s also important to keep your survey short and snappy and tell would-be participants how much time it will take. The shorter it is, the more people will likely take part. If you’re using an online survey, you might want to add a progress bar so that respondents know how much of the questionnaire is left. That’s a simple but effective incentive to complete the survey. 

And here’s one last tip. Surveys can be built around “open-ended” or “closed” answers. Generally, it’s a good idea to keep the former to a minimum and focus more heavily on the latter. Why? Well, answering open-ended questions takes more time and thought than responding to simple “yes or no” questions. The easier the survey, the greater the chances participants will finish it.

What comes to mind when you think of user research? Chances are, it’s a company asking what customers think of its latest technological gizmo. Sure – that is a pretty common question. But user research also tackles more sensitive issues. In this case, it’s important to proceed carefully and tactfully. 

Take it from the author. In 2014, she was conducting user research for a charity that helps people with terminal illnesses and their loved ones. Most of the folks she was talking to were either terminally ill or related to someone who was. As you can imagine, emotions were running high. 

The author understood that this was a situation in which participants’ well-being trumped her job of gathering data. That’s why she started each interview by reminding people that they could stop at any point. If she noticed that a participant was becoming upset, she gently brought things to a close herself and made sure a counselor was on hand to offer support. 

How you structure your interview sessions also makes a difference when it comes to delicate subjects. If you’re talking about something personal, it’s generally a good idea to take it slow, start with easy questions and build toward more complex issues. 

One way of doing that is to kick the session off by explaining the context of your research and giving participants a chance to ask you questions. Once you’ve done that, you can move onto warm-up questions of your own. These are designed to help you get to know participants a little better. Keep it vague at this point, and try to find out how much experience or knowledge your subjects have of the topic you’re exploring. Once you’ve put participants at ease, you can move on to more sensitive and personal questions. 

User research isn’t just about gathering data – it’s also about communication. Participants won’t always give you the unambiguous and straightforward answers you need. Luckily, there are a few tricks you can use to get through difficult interviews.  

Let’s start with awkward silences. There are few things interviewers dread more, but they’re part and parcel of user research. That means you should accept them, rather than filling them with your own words. Why? Well, improvising and making conversation is usually counterproductive, since it derails the conversation and steers you away from your research topic. 

There’s also a risk that you’ll end up asking leading questions, thus compromising your neutrality. Remember, your data is only useful if it reflects what people actually think. If you’re pushing people to say something, chances are their response won’t reflect their true opinion. 

So instead, try to ride out those awkward silences. Sooner or later, participants will fill them. You can encourage them, but ultimately you’ll want to wait to see what they say without your interference. If a participant is truly tongue-tied, you can talk to them, but this should be an exception rather than the rule.

Fielding questions is another aspect of interviewing that can be difficult. When deciding whether to allow yourself to be drawn in, it’s useful to identify what kind of question you’re being asked. If a participant isn’t sure what you mean and asks you to clarify, you can rephrase your question. But if they ask you what you think, you should resist commenting. Diplomatically remind them that you’re interested in their opinion and tell them that you’re happy to share your point of view after the interview. 

Like fish in water, humans usually take their environment for granted. We function on autopilot and don’t question our immediate surroundings. That means you’re unlikely to gain any real insights by asking interviewees how they feel about the world around them. That’s where ethnography comes in. 

Ethnography is the study of people and culture. It involves in-depth observation of how people behave and interact with their surroundings. As an ethnographic researcher, your aim is to reveal things you may have missed by failing to consider the real-world application of your products or services. 

Take airports, for example. Initially, planners wanted to install electric sockets in the walls near the ground. That makes intuitive sense – after all, that’s where sockets go in most buildings. An ethnographic approach, however, shows that this is far from optimal because it means travelers have to sit on the floor to charge their devices. If you’ve ever wondered how anyone came up with the great idea of incorporating sockets into airport seating, there’s your answer!

Modern technology makes this kind of research simple. Because the ethnographer doesn’t have to be present, researchers can practice so-called “mobile ethnography.” That’s useful because on-site ethnography isn’t practical in every situation. Like moderated usability research, it requires the researcher to be somewhere at a certain time.

Today, however, the latest tech devices mean that participants can record their own daily activities, as well as their own thoughts and feelings. Mobile ethnography is essentially about reviewing and analyzing that self-collected data remotely. This way, it’s less intrusive, and folks can participate in your research wherever and whenever is convenient for them.

Now you know about the different user research methods out there. That means you can start finding out how people feel about the service, product or experience you’re offering. Once you’ve figured out which method best suits you, you’re ready to dive in and gather that most precious of business resources – data.