Conduct icon usability testing



Business Benefits

Optimize the icons on your site to improve user experience.

Test the placement of your icon using time-to-locate tests, to evaluate how easy your icon is to find in-context.

Ask yourself the following questions and conduct heuristic analysis:

  • Does it blend in with other icons?
  • Is it where it is expected to be?
  • Is it being overshadowed by other page elements?

Next, conduct time-to-locate tests by asking people who are unfamiliar with your site or app, to perform a specific action.

Then, measure how long it takes them to find the right icon for the action and how often they choose the right icon on their first attempt.

Test to determine if your users can identify your icon out of context and without labels.

Ask yourself the following questions and conduct heuristic analysis: can people identify the symbol?, how quickly can they identify it?, and can they identify it on mobile?

Next, ask other people the same questions. Show them the icon and ask them to explain what they think it is. For example, they may think the icon is a trash can or a calendar.

Test to find out if your users can comprehend your icon out of context with some clues.

Ask yourself the following questions and conduct heuristic analysis:

  • Do visitors know what will happen when they click?
  • Do they associate the icon with any other action?
  • Is the icon familiar?
  • How does the icon come across on mobile?

Then, ask other people what they believe will happen if an icon is clicked. For this question, you need to provide some information about the website.

You can conduct an A/B test to find the icon with the best scent. The icon position and label should stay the same in both variations you test. If visitors click the icon and then quickly return to the previous page, you know that the icon has poor scent.

For example, Etsy used these two multi-use icons:

Etsy's Hamburger

Etsy's Heart

Multi-use icons create confusion. To evaluate how comprehensible these Etsy icons are, ask the following questions:

  • What would you guess that they do?
  • Is the top icon a menu or a list?
  • Is the heart a like or a save to wishlist?

Test to see if your icon is necessary using an A/B test to help your visitors in context.

Ask yourself the following questions and conduct heuristic analysis:

  • Is it the right time to use an icon?
  • Can you use text instead?
  • Is the icon actually helping your visitors?

For example, you can conduct an A/B test by leaving the position the same and in variation one, use an icon. In variation two, use plain text. This will show you whether the icon is necessary in the first place and whether it actually improves the usability metrics.

Test to see if your icon is visually appealing out of context.

Ask yourself the following questions and conduct heuristic analysis:

  • Does my icon look good?
  • Does it look good with the other icons on the site?

For example, you can have people score your icons on a scale of 1-5. Alternatively, you can simply ask them to choose their favorite and explain why. What do they like? What do they not like? Use these insights to design visually appealing icons.

Conduct tests multiple times and phrase the tasks differently each time, to avoid creating imaginary associations.

For example, a participant can be subconsciously primed to associate a heart with biology due to the way the question was worded. This invalidates your test results. By conducting tests multiple times and phrasing the tasks differently each time, for example, using synonyms, you ensure the wording is not influencing responses.

Tailor your icon testing to the current stage of development.

In the earliest stage, you should be testing whether your icon is a good idea. At this stage, you are focused on out of context tests. You can ask, can people identify and comprehend these icons? How easily?

When starting development, you should be running multiple tests. Start with out of context visual appeal tests. Then, move to in-context tests that gauge how easy it is to find your icons.

After development, focus on optimization. Start running A/B tests on the size, position, symbol, and order of your icons, to make marginal improvements over time. Continue testing the five elements by asking How do the scores improve over time?. If they do not, begin again, and conduct standard usability testing during this time to confirm your findings.

Test the specific elements of your icon and focus on one minor element per test.

Avoid using standard usability tests blindly to try and determine whether an icon is effective. Standard usability tests should support your other findings.

For example, you run a standard usability test to see if your icon is effective, your first participant does not even notice the icon. She was distracted by another element of your site. The second participant stays longer, browsing the site normally but also does not interact with your icon either, and you have no idea why. You do not know because there are too many factors involved.

This type of testing does not work because it does not answer the following questions:

  • Did your participant even see your icon?
  • Was your icon ugly?
  • Was your icon confusing?
  • Was your icon unnecessary?
  • Was your icon hard to identify?