Business benefits
Create a brief from the Speero usability study template.
Define the user recruitment criteria and decide how many users to recruit.
Audience criteria
- Is the audience generic or specific? For example, pretty much everyone wears shoes; very few people operate ultrasound machines. A B2C audience is often generic; a B2B audience is often a lot more specific.
- What are the demographic criteria? Age, gender, etc.
- What are the key inclusion criteria? For example, they must be in the market for a new x, they must be working in x role or industry, or must use the internet to shop online at least once a month.
- What are the exclusion criteria? For example, must not be a UX designer, must not be a brand or product rejector, or must be open to online shopping.
- Choose an age range for targeting based on information from your website analytics - do you mainly have a younger or older audience?
- If your product is very specifically marketed at one gender, limit the target users to that gender; otherwise leave the targeting open.
Number of users
- As a general rule, use at least 5 users per segment. If your only segments are desktop and mobile users, then you need 5 for desktop; 5 for mobile. The more segments, the more users you’ll need.
- Check analytics data to determine how you should split the sessions across different segments. For example, if 80% of your traffic comes from mobile, run more sessions on mobile compared to desktop.
Pick recruitment options and, if your targeting includes inclusion or exclusion criteria, write 2-3 screening questions.
If you’re looking for a very niche audience in a short time, consider sourcing participants through an on-site poll or emails. Alternatively, use a crowdsourcing platform or specialist research recruitment agency.
If you’re looking for a wider audience, or you have longer to find your users, it’s simpler to use a platform like Userlytics or Usertesting.com.
Use screening questions for niche audiences or if the standard targeting options will not be narrow enough to match your ideal users.
Tips for writing an effective screener:
- Think of your screener as a funnel that filters out unsuitable users quickly. Use a staged approach that starts by asking broad questions and gets more specific. For example, Which of the following best describes you? followed by How often do you do this?
- Consider both inclusion and exclusion questions - think about the people you definitely don’t want to include, as well as people that you do. For example, if you were recruiting for a website that sells gun holsters, you wouldn’t want to recruit someone who was against gun ownership.
- Ask open questions so users don’t know what the right answer is. For example, ask Which of the following household products are you planning to purchase in the next 3 months? instead of Are you planning to purchase a new sofa within the next 3 months?
- Think in terms of internet knowledge and topic knowledge. Do you need frequent internet users? Do you need people who are in a specific role, or in the market for a specific product?
- Limit yourself to 2-5 screener questions. Focus on behaviors rather than demographics you can catch with targeting.
Example screener for men’s formalwear Ecom website:
Write tasks that are clear and realistic, without being too specific
In general, avoid outright telling the participant to do something, as this doesn’t give us insight into what users would do naturally. Start broad and then get more specific.
For example, if you want feedback on a homepage element that is below the fold. Don’t ask users to scroll down and tell us what you think of x element. Instead, first, observe user behavior – does the participant scroll down without a prompt? Either way, this is interesting. Then make the next task more specific. Takeaways might include that although participants were positive about the element, none of them would have seen it in the real world, which could cue a test idea to increase the prominence of this element.
User goal: Browse product offerings and purchase an item.
Poor task: Purchase a pair of orange Nike running shoes.
Better task: Buy a pair of shoes for less than $40.
Other example tasks:
- Google task: To emulate a natural user experience, begin with an exploratory task. This can gather insight around acquisition strategy, help you better understand the wider user journey, and ease users into the research session.
-
Find an x product: For ecommerce, ask participants to find a product with a specific price range, material, or other specifications. Give the participant time to choose the product and observe their product-finding process. This helps you observe the usability of the filters and site navigation.
For SaaS, ask participants to describe what the product offers and how it can help them. What are the benefits and USPs? - Compare products or plans: Ask participants to compare two products or plans and choose one suitable for them. This will help you understand what visitors will notice, and their thought process.
- Add-ons or upsells: Ask participants to add items to the cart or buy something extra with their order. See how easy they find the process (observe if they do this without prompt first).
- Registration or checkout: Ask participants to go through the signup or checkout process to allow you to identify issues and friction points.
Write 3-5 pre-session questions that offer context about the user’s feedback, and 2-3 post-session questions that offer data about the overall UX of your website.
Tips for writing the pre-session questions:
- Keep the questions to a minimum – 3-5.
- Look at the questions from the customer survey for some inspiration.
- Find out more about their fears, uncertainties, doubts, and motivations in this area, e.g. what are your key considerations when purchasing sportswear online?
Example pre-session questions for a niche client
-
Please tell me a bit about your role as a healthcare professional.
- What is your role?
- How long have you worked in the industry?
- What does a typical day look like?
-
What is your experience with ultrasounds or ultrasound technology?
- Do you conduct ultrasounds personally?
- Does your team conduct ultrasounds?
- Are you responsible for researching or purchasing ultrasound technology for your team, clinic, or network?
-
What are your main considerations when researching or purchasing ultrasound technology?
Example pre-session questions for a ecommerce client
- Please tell me a bit about your shopping habits.
- Do you purchase clothes every month, every 3-6 months, once a year or so?
- Where do you usually shop – online or physical shops?
- What type of clothes do you usually buy online?
- Do you mainly look for office shirts, casual clothing, or suits?
- What are your main considerations when shopping for clothes? What do you care about?
- Are you looking for free shipping, quality fabric, or best deals and offers?
Examples of post-session questions
- Did you have any questions or was there anything you would have liked to have known before completing the purchase?
- How would you describe your overall experience with the product or website?
- If you could change one thing on the website, what would it be and why?
Set up the sessions on your usability study tool of choice.
General recommendations:
- Don’t give away screening information in the public-facing name for the user test.
- Add enough information to the internal name to ensure that it’s easy for people to identify weeks or months later.
- Launch 1-2 sessions first, watch them back and then tweak questions, tasks, and screeners if needed. Sometimes users misunderstand questions or the sessions don’t flow quite as expected.
Launch the sessions with your chosen participants.
Use one of the preview links to test out the flow before you launch.
Watch back the videos and make notes on what participants say and do. Code and analyze the video recordings using the Speero usability study results template.
Don’t just record verbal feedback – behavior and actions are very important as well. Include the elements that they click on and any hesitation, confusion, or frustration you notice – and where it occurs.
Highlight anything that seems to be broken, not working well, or causing users confusion, frustration, or difficulty. Highlight those comments with different font colors. For example, use red for any problem areas, orange for opportunities, and green for successful tasks or positive verbal responses.
If something is not working well or there is a clear issue on the site, note down how many participants saw this problem and include that information in the slides later.
Add insights to slides to use in reporting to stakeholders. Add screenshots, quotes, or examples for each issue you mention.
If you see issues that are difficult to describe in words, crop videos into short clips and embed them in the slides. Keep these videos as short and snappy as possible.
Things or insights to look out for when creating the slides:
- Benefits: What did the participants like about the site? What is working well? – color-coded with green (overall this is the lowest priority)
- Issues: Any bugs, technical issues, UX difficulties or points of friction that the participants faced, questions that didn’t get answered – color-coded with red.
- Opportunities: Larger scale ideas that could solve a problem or improve the overall on-site experience. For example, the participants suggested a savings calculator to compare their current electricity plan to yours – color-coded with orange.
When putting together a slide about an insight, add:
- Summary of the issue.
- Supporting quote.
- Screenshot or video of the issue.
- Mention how many participants had this issue.