Select Page

Usability Evaluation Methods: When to use them & what they require

by | Apr 6, 2022 | UX optimization

What is Usability Evaluation?

Usability evaluation is the practice for evaluating how successful a web or app product is in helping its target users get what they want out of it: whatever that may mean.

There’s two steps to the process:

Data collection: Finding data about your web or app’s “usability”, be it quantitative or qualitative

Data evaluation: Turning that data into actual insights you can use and keep in mind when you build new designs, layouts, functionalities, user flows etc.– with the ultimate hope for what any digital designer or UX professional wants: that their users end up finding it “easier / better / clearer ” to use the web or app in question as a result. 

What exactly is easier/better/clearer? Well, then we go back into usability. So let’s define what it is.

What makes something “usable” (or non-usable)?

Usability, though a pretty popular buzzword in this day and age, is a pretty intangible concept because it doesn’t talk about one particular factor, it talks about a whole group of factors that attempts to answer the following questions.

The more the answers are yes, the higher the “usability”.

  • Is the design intuitive and easy to scan through?
  • Do users get a feel that the design is relevant and apt for the topic of the product?
  • Does it help users get through the actions they need to take clearly?
  • If a user that’s never seen that UI before arrives there, would he be able to navigate through in his first attempt?
  • How often do users end up behaving strangely on that application, finding themselves with error messages?

All these questions are what you can hope to answer with a usability evaluation. And that is what we’ll focus on next, method by method:

Usability evaluation methods – best situation to use, and what they require

1. Usability testing

Usability testing and evaluation may sound like the same thing, but they’re not. Usability evaluation is the “umbrella term” for all we’re talking about this in post.

Usability testing, however, is when you specifically have a group of people (a “focus group”, if you will) go to your web/app product and observe how they go through and interact the user interface, and jot down insights on how easy and clear they found it to use. Of course, the goal here would be to find improvement opportunities.

There’s two sorts: moderated and unmoderated. In a moderated study, someone working for you can encourage the users in the test to share more about what the they’re doing and note feedback. In an unmoderated study, you can ask a user to think out loud and record them with their consent, but there is no one there to remind her if she doesn’t do it (hence unmoderated).

Best situation to use: When you need to ensure your website or app meets the expectations of your users, after development phase and before deployment.

What it requires: 5 to 20 users in a focus group to participate in your study, ideally ones that can be representative of your customers (in terms of demographics).

2. Heatmapping

Heatmaps in UX are a visual representation of how users behave with your digital interface, aggregated into one graphic that shows:

– Warm areas: that get a lot of attention
– Cold areas: that, well, don’t.

There’s multiple kinds: click maps, scroll maps, and hover maps.

Feel free to read my other blog post on it if you’re interested to find out more.

Best situation to use: When your traditional analytics don’t offer you enough insights on what’s happening on your web-page and how users interact with it.

What it requires: A heatmapping tool, such as Hotjar, Smartlook or CrazyEgg.

3. Card sorting

This is another good evaluation method to use if you want to find out if your UX is built on a house of cards. (See what I did there?)

It’s an old school method but still pretty popular.

In a card sorting session, people from within your organization help organize topics into categories (thus making, “category trees”) that make sense in their context and with their different expertise. Then, you want to end up with some form of a structure of how to organize different parts of your websites and where they belong in their categorization.

 Best situation to use: When you’re strapped for budget but plenty of colleagues in different roles that can all bring different perspectives in your research.

What it requires: Either a card sorting software, or just plain old fashioned post-it notes.

4. First click tests

Web analytics like Google Analytics can share a pretty clear picture of what steps a user has taken, but what they can’t tell you with certainty is what the user was actually trying to get out of a click.

That’s where click tests can come in: in these, you give the study participants a task (e.g. where would you click to do X) and you see where they actually click. Voila! Then you can unlock some golden insights into the most important goals of your usability.

Best situation to use: When you don’t want to dig too deep but find a low hanging fruit that can make a drastic impact.

What it requires: Either a user testing platform where you study this behaviour in the research, or setup this information capture in an analytics tool for live testing.

5. Diary studies

Diary study is when you take a “hello diary” approach over time to evaluate your UX.

It’s a qualitative UX evaluation methodology, where you gather user experience logs (hence diary) over a longer period of time. The participants of the study would write about their experiences about your product or service over time, and once it’s over – you can gather it all up, analyse and draw some findings. It’s as simple as that.

Best situation to use: When, again, you’re limited on budget and want insights from an extended period of time rather than a one-off snapshot.

What it requires: Users that can participate and write a journal for you. It’s also smart to come up with some ideas on kind of comments or thoughts (i.e. where to focus) you want them to write.

6. Session Recordings

Another qualitative form of usability evaluation, where you just silently (and somewhat creepily) watch how your users behave on your web or app in the form of session video recordings.

Now, before you start to worry about how creepy it is: the recordings would only show how your users behave on your digital interface in the form of mouse movements and keyboard interactions, so you can watch them navigate through your interface and see if they run into usability issues in trying to achieve what they want out of their interaction with you. Most tools have the capability to protect user input from keyboard by showing it as asterisks.

You can also usually mark tags in these tools that offer this, so you can note how users behave and use tags to highlight the interesting recordings and save them for later.

Best situation to use: When you want to spot bugs, follow users behaviour closely and have the time to go through hundreds of videos and tag.

What it requires: A session recording tool like Hotjar or Smartlook.

7. Guerilla Testing

Guerrilla testing (or as some call it, “hallway usability testing”) is probably the most informal approach we have on this post – but still a valid enough method to test ideas and get some superficial feedback.

All you do is go up to random people and ask them for their feedback on your product. (I know, pretty complex.)

Of course, you can narrow it down in certain ways by trying to target certain demographics of people that you believe fit the representation of your users.

Best situation to use: When you want to go very basic and use an offline methodology to improve your UX with little to zero costs.

What it requires: Just someone to go around asking people questions!

8. Polls and surveys

And finally, one of the easiest methods – good old fashioned surveys and polls.

Two of the easiest approaches are:

  • Email a survey asking about your interface to existing customers or contacts that can help give you feedback (good idea to add incentives for the participants!)
  • Set up an on-page website poll: where users can leave feedback while they’re browsing your product.

It’s particularly great to combine with other forms of evaluation methods. Make sure to craft a clear survey that asks questions crafted in a way that can help you uncover the most useful insights to improve your product, though!

Best situation to use: When you have very specific questions you want answers to.

What it requires: 1) Existing inflow of traffic, and 2) A website polling or survey tool.