As a product designer at Five & Done and former data scientist/ML engineer, I’ve been involved in some projects in which I’ve had to use the analytical left part of my brain to help run user studies, interpret A/B test results, and come up with research plans.
User research is a critical part of the design process at Five & Done and we do this to confidently create and launch products that will achieve their desired objectives. It’s inefficient and impractical to prematurely release things without any testing for the sake of being first. Do you remember the failed product launch of Google Glass back in 2013? While it was cool and all, introducing people to the world of Augmented Reality, it was discontinued in 2015 because Google hadn’t properly addressed user pain points and other usability issues.
We can’t fail our clients and if we were to launch something without going through at least one of these efforts below, we’d simply be hoping and wishing for success.
In this post, I’ll cover how we use different user research methods at Five & Done to best serve a project.
But first, a couple of things -
- This isn’t a ‘how to implement each method’ article; it’s more of a ‘when to use each method’ type of piece.
- I’m going to try something else and give a set of 3 principles I’ve found to be helpful when it comes to evaluating each research method.
- There are TONS of ways out there to do user research - I’m only focusing on the ones I’ve found to be the most efficient and valuable for us and our clients.
Let’s get into it!
Three Criteria To Evaluate Each Method.
1. Product Stage
Is your product an existing one or an idea locked away in your brain that hasn’t made it to paper yet? The phase is important to consider because that will dictate the types of insights and research goals you have. For example, if your product is in the earliest stages, you may be more interested in testing out the concept with a couple of target users, instead of evaluating the performance of a new design.
2. Research Question(s)
What are you trying to measure? Is it important to know the statistical significance of certain things? Do you want to know the ‘why’ behind user decisions, or are you more interested in knowing ‘what’ & ‘how much’? The type of insight you want will greatly influence what method(s) you choose.
3. Complexity
How complex is the method? Some are quick and easy, while others need more time, tools, or expertise. Consider how much effort and resources it takes—simpler methods are faster and need less analysis, but more complex ones might give deeper insights.
Our Top Research Methods, Let’s Go!
1. Surveys
A series of questions is sent out to a target audience to collect quantitative feedback. This is one of the most popular methods because it reaches a wide range of users.
Product Stage:
- You want initial information about your target audience, their demographics, and their opinions at the start of a project
- You need to measure the success of a recently launched product
- You need to measure satisfaction overtime
Research Question(s):
- What are the demographic qualities of my target audience?
- What are major pain points in the current user journey and how widespread are they?
- How do users feel about the current product? How often do they use it? What do they use it for?
Complexity:
Low. It’s relatively fast to create, gather data, and analyze results. The downside is that responses may lack context because you can’t ask follow-up questions to cover more depth.
2. User Interviews
An open-ended method designed to get qualitative feedback on a product or topic of interest, in which an interviewer will ask a user several questions about their experience, motivations, & habits in a 1-1 discussion.
Product Stage:
Typically during the exploration of an idea, problem, or opportunity space.
Research Question(s):
- Why do my users experience certain pain points? Why did they make certain decisions?
- What does the current user journey look like in detail?
- What are my users’ motivations and habits?
Complexity:
Medium. A more time-consuming method that requires highly skilled interviewers, and depending on the number of interviews you conduct, it can take a while to both run & analyze the sessions. However, you get rich, qualitative insights that explore the ‘why’ behind user behaviors.
3. Focus Groups
This method is similar to user interviews other than the setting, in which small groups of selected participants will be asked to openly share their feedback on a set of topics or a product. The group discussions are facilitated by researchers.
Product Stage:
Usually concept or early design stages
Research Question(s):
- Can my users be grouped into distinct groups when it comes to their underlying motivations?
- What are your opinions on new product concepts or the existing market?
- What pain points are being brought up the most?
Complexity:
Medium. Depending on the number of participants, focus groups could be more time-effective than doing several 1-1 user interviews. They’re also a great way of collecting diverse perspectives because of the encouraged discussion. On the flip side, there’s a higher chance of bias because of groupthink and unless you have a skilled interviewer, it can be hard to manage group dynamics.
4. Usability Testing
Usability testing involves observing real users complete tasks as they interact with a product to identify any issues and gather feedback on how easy & intuitive the product is to use. These sessions can be moderated or unmoderated. There are different metrics to assess how well a design performed, such as time on task, number of errors, system usability scale, and net promoter score.
Product Stage:
You have a prototype or a fully developed product, and you want to test it at different stages to guide new designs or improve the current product.
Research Question(s):
- Are my designs easy-to-use and let users complete the tasks at hand?
- How many users were able to successfully complete tasks error-free?
- What are the most common pain points in my prototype?
Complexity:
Medium. If it’s a moderated session, it’ll be more time-consuming to both run & analyze the sessions; if it’s unmoderated, more time will go into post-study assessment. A lot of it also depends on the total number of participants. This method is very important to conduct multiple times if you can throughout your design process because you’re getting direct feedback on how users interact with your product, which can reveal usability issues.
5. A/B Testing
A quantitative test designed to compare 2+ versions of a web page or app to see which one performs better. This helps optimize design and improve the user experience based on data-driven insights.
Product Stage:
You have an existing, launched product with enough user traffic to create statistically significant test samples.
Research Question(s):
- Which design drives more engagement and results in things like higher conversion rates, higher click-through rates, or lower bounce rates?
- How can I evaluate experimental new features to a subset of users?
- How can I test iterations on CTAs, button colors/labels, new user flows, etc.?
Complexity:
High. Typical A/B tests need a large user base to be effective, you need to be careful of preventing outside factors from influencing results, and these tests can run for up to a couple of months to ensure enough data points are collected over time before choosing which design to go with. A lot also goes into making A/B tests work, since design, engineering, product, analytics, & other teams have to all be aligned, and you have to ensure that designs and analytics tagging have been built properly in order for the test to run. But what you get in the end is hard, indisputable evidence that proves whether or not your designs were effective.
6. Concept Testing
This involves presenting very early stage ideas or prototypes to target users to gather feedback on their appeal and usability.
Product Stage:
You have a very early concept (or possibly two different ones) that is probably still on paper, and you want to test the direction before continuing with further development.
Research Question(s):
Are my target users buying into this concept and product goals?
Complexity:
Low. There’s a quick turnaround in running/analyzing sessions and iterating on feedback. A lot of times, concept testing can be done guerilla style, meaning you simply take your low-fidelity design or prototype out to the general public and ask whoever you can for feedback and validation on whether or not your idea works. For that reason, the best thing I find about concept testing is being able to validate ideas early, which saves us a lot of time and resources by testing before building. The one drawback is that because this method is done before you have anything that’s high fidelity, you don’t capture real-world use, and it relies on user imagination of what the end product could be.
7. Card Sorting
An activity in which users are asked to use digital or physical cards to sort information into categories based on what makes sense to them.
Product Stage:
You are in the early discovery phases of your product.
Research Question(s):
- How do users prioritize, sort, and classify a set of items?
- Which features do users want most or first?
- How should I create an intuitive navigation system and information architecture for my website or app?
Complexity:
Low. Relatively fast and easy to conduct, these sessions are generally quick and the type of data gathered is easier to assess. However, although the point of this method is to help structure your information hierarchy, you’re limited to understanding how to organize content and can’t test actual functionality.
8. Content Testing
A study aimed at understanding the messaging, usability, and clarity of content in your designs. Content testing helps reveal if users can easily find and understand important information. It also shows if the tone/style makes sense and if the text is easy to digest, or if you should use images instead of short video clips for a product page as an example. If users don’t understand your product’s value or find the content too jargon-heavy, they’re less likely to buy.
Product Stage:
You have content ready to be reviewed but still flexible enough to be adjusted—usually in the later stages of development or when refining specific features, new pages, or messaging before a product launch or update.
Research Question(s):
- What type of content best shows the benefits of a product or service?
- Is the content easy to follow?
- Does the content tone and style appeal to my target demographic?
- Is the content relevant to user needs and supports the overall user experience?
- How does content affect usability and task completion?
Complexity:
Low - High. The beauty behind the principle of content testing is that there are various methods you can use to conduct it - usability testing, surveys, or A/B testing, to name a few. In terms of what you’re assessing, it’s straightforward since you’re getting to understand if the content is clear, effective, and easy to run. Keep in mind that you’re not assessing overall designs.
Wrapping It Up
Depending on where your project fits into these three criteria, the methods you choose can be used individually or as a combination like how we did for a recent project detailed here in this blog post! With this foundation, I hope you feel equipped to kick off your user research journey, whether solo or with your team. Have fun with it!
.png)
.png)