Use surveys to measure user satisfaction

Business Benefits

Measure usability and the overall user experience.


Run a task-level satisfaction survey immediately after users complete a task to determine how difficult or easy it was to complete the task.

  • ASQ uses three questions about the ease, time, and support received.
  • NASA-TLX assesses the workload involved, including weightings for each of six elements.
  • SMEQ uses one scale to determine how much mental effort a task requires, and correlates highly with SUS scores, completion time, completion rates, and errors.
  • UME allows users to assign a number to a task to estimate its difficulty in proportion to the perceived difficulty. For example, a task perceived as 100% difficult is twice as difficult as a task perceived as 50% difficult.
  • SEQ asks users to rate the task on a scale from very difficult to very easy.

Assign a number to the task experience, such as number of problems encountered or number of steps to complete a task, to see where you can improve.

Improving this number over time allows you to measure how designs have improved the UX, and then fix the bottlenecks and improve conversions, thus improving revenue.

Run a test-level satisfaction survey at the end of the session to measure the user’s overall impression of the usability and experience.

  • SUS consists of 10 statements to rate on a scale of 1–5:
    • I think that I would like to use this system frequently.
    • I found the system unnecessarily complex.
    • I thought the system was easy to use.
    • I think that I would need the support of a technical person to be able to use this system.
    • I found the various functions in this system were well integrated.
    • I thought there was too much inconsistency in this system.
    • I would imagine that most people would learn to use this system very quickly.
    • I found the system very cumbersome to use.
    • I felt very confident using the system.
    • I needed to learn a lot of things before I could get going with this system.
  • SUPR-Q has 8 questions (including 1 NPS question) to determine if a website is usable, credible, and visually appealing. The trust questions vary based on whether the site is commerce-oriented.
    • The website is easy to use. (usability)
    • It is easy to navigate within the website. (usability)
    • I feel comfortable purchasing from the website. (trust for commerce sites)
    • I feel confident conducting business on the website. (trust for commerce sites)
    • The information on the website is credible. (trust for non-commerce)
    • The information on the website is trustworthy. (trust for non-commerce)
    • How likely are you to recommend this website to a friend or colleague? (loyalty)
    • I will likely return to the website in the future. (loyalty)
    • I find the website to be attractive. (appearance)
    • The website has a clean and simple presentation. (appearance)

Score the SUS to use as a comparison to other products.

  • For odd items, subtract one from the user response.
  • For even-numbered items, subtract the user responses from five. This scales all values from 0 to 4, with 4 the most positive response.
  • Add up the converted responses for each user and multiply that total by 2.5. This converts the range of possible values from 0 to 100 instead of from 0 to 40.

Score the SUPR-Q to use it as a comparison to industry benchmarks.

Add up the responses for the first 12 questions. To that total, add one half the score of the NPS question. The lowest possible score is a 12, and the maximum score is a 65.

Ask users, How likely are you to recommend the product or service to a friend?

This is known as Net Promoter Score (NPS) and is a popular and effective method of measuring user experience and satisfaction. Break up NPS responses into three user segments:

  • Promoters (9–10). The happiest and most loyal customers, most likely to refer you to others. Use them for testimonials, affiliates, etc.
  • Passives (7–8). These customers are happy but unlikely to refer you to friends. They may be swayed by a competitor fairly easily.
  • Detractors (0–6). These are unhappy customers and can be dangerous for your brand, spreading negative messages and reviews. Identify their problems and fix them.

Subtract the percentage of detractors from the percentage of promoters to get the NPS score. Use the NPS score as a benchmark for how well you are doing. A good NPS is better than the one you had last month.

Compare your numbers to competitors to get a realistic idea of your customers’ satisfaction level, and then you can work on improving your score.

Investigate trends that are improving or worsening.

Plan targeted promotions and campaigns by knowing more about your promoters.

Your promoters will help tell the world about you.

Use the detractors to determine what you can improve about your user experience.

  • Which issues keep appearing?
  • Are there any trends?

Last edited by @hesh_fekry 2023-11-14T15:50:32Z

Hi Alex.

I have a few questions.
Is the missing context for this playbook that these surveys are for a SaaS customer/user?

Point 1 and 3 - What’s the difference between task-level and test-level satisfaction?
Point 3 - Would you run a test-level satisfaction survey at the end of every session?
Point 7 - Is there software/service that you could use to compare your numbers to competitors - what numbers are you comparing?
Thanks.

1 Like

Hey @luke.thompson.alex great questions.

This playbook was based on 8 Ways To Measure Satisfaction (and Improve UX) by @alex_birkett.

So I believe the focus here is on UX, the blog does not specify if its SaaS specifically but the examples like Slack seem to be.

  1. The blog states:

If task-level satisfaction is measured directly after each task is completed (successfully or not), then test-level satisfaction is a formalized questionnaire given at the end of the session.

  1. We tend to do this yes as a general rule. Depending on the length of the test or the goal we would use SUS/SUPR-Q or Net promoter score

  2. There are websites like Comparably which you can use, or research for benchmarking articles and data to base this on. It will really vary depending on what industry your in and who exactly you are looking for as the degrees of which they share/collect/publicize this may vary a lot.

Did you have a specific field/industry in mind?

1 Like

Hey @hesh_fekry - great to hear from you again!

I loved the blog post - it put a lot earlier questions I had about the context into perspective.

Particularly point 9. - it was really interesting hearing about how the broader debate around Net Promoter Score (NPS) and how rather than using the data as an industry comparison point it suggests which customers to focus attention on.

I don’t have a specific industry in mind. I find that regional companies in Malaysia (where I’m based) are underrepresented in databases/aggregators.

Thank you so much for addressing each point I had!

Is there any way that I can value-add to the playbook community that would be of help to you, personally? Thanks!

1 Like

@luke.thompson.alex

Thats great to hear.

The best value for us and for the other member of the community is to ask and answer questions just like you have done now. This helps to poke holes in the content, refresh and make the playbooks more useful by helping us iterate on the library with more focus.

Which areas of marketing are you specialized in or most interested in?

@hesh_fekry superb!

I love poking holes - always happy to iron out content.

At the moment I’m looking at learning more about e-mail marketing - I’m currently looking at the excellent work Jessica Best did in the foundations course but it’s a little difficult to put her advice into practice. Enter the playbook community, haha!

In terms of upskilling and future-proofing myself I’d love to get more insight into either growth marketing and/or CRO!

What first drew you into this field? Thanks!

1 Like

Hey luke, great to hear from you again.

Well well well, you may like these:

The playbook project as a use case I found super interesting to be honest. I am a trained Architect by trade. Teaching people do things in a new way is a long way from the traditional practices i was used to. Super exciting.

Let me know if you need some help finding resources on growth and CRO?

1 Like