5 reasons why your first user research activity should be a usability test

A usability test is the wrong research method when you want to discover if there's a real user need for your product; when you want to understand the environment where your system is used; and when you want to find out how people use your product in their daily lives. So why do I almost always recommend a usability test as a team's first user research activity?


User research is, in part, a scientific activity: you generate hypotheses and test them by collecting data. Then you revise your hypotheses in light of your findings. But unlike science, user research is also a political activity. For example, some organisations treat user research, and indeed the whole field of user experience, as merely a passing trend or fashion. Some managers need to be convinced that user research is needed. And some development teams just don’t understand what user researchers do all day. 

The politics of user research may also explain why development teams often prefer to start their user experience journey by commissioning an expert review. An expert review doesn’t involve users: instead, a handful of usability experts evaluate the design against good practice. I think expert reviews are attractive to design teams not simply because they are quick and cheap but because they don’t require the team to engage with users: the team can claim they have ‘done usability’ when in fact there were never any real users involved. It is also easy for the team to dismiss any critical findings from an expert review as being ‘just the view of the consultant’. Ignoring the behaviour of users is much more difficult.

Given that ‘user research’ implies we need to involve users in our research, the most (scientifically) logical user research activity is a field visit — especially in the early stages of design. This gives you the opportunity to test out your hypotheses around user needs. But if you find yourself in the kind of environment I’ve described above, your decision should not be based on logic but on emotion. You need to answer the question: what user research activity will have most impact in making this organisation user centred?

And in my experience, the answer is often a usability test, for 5 reasons:

  • You’ll identify the business objectives.
  • You’ll discover the key user groups.
  • You’ll reveal the key tasks.
  • You’ll flush out the stakeholders.
  • You’ll establish if there’s an appetite for user research in your organisation.

You’ll identify the business objectives

To run a usability test, you need to know the business objectives of your product because otherwise you don’t know where to focus your test. What is the business trying to achieve with this thing? It’s not unusual for different business units within an organisation to hold competing, contradictory or inconsistent business objectives. So getting a direct answer to this question is absolutely crucial for any future user research and design activities you engage in. To make sure you have this clear in your own head, make sure you can answer this question: “If we didn’t run the test, what would be the main business risks?”

You’ll discover the key user groups

Product teams are often dysfunctional. It’s common for them to suffer from a kind of groupthink where no-one really knows who will use the thing they are designing — but at the same time no-one wants to admit that this is a problem. There’s a general belief that someone will use this thing, otherwise the project would be shut down. To run a usability test you need a clear description of the main user groups, otherwise you won’t know who to recruit for the test. And in the same way that a psychotherapist needs to make a dysfunctional family confront the elephant in the room, you’ll do the same when you plan your test.

You’ll reveal the key tasks

Design teams maintain a backlog of technical work that they want to complete and when this gets overly long you’ll find someone adds ‘simple to use’ to the list. But ‘simple’ isn’t a feature. You can’t build ‘simple’ into a product. You have to weed complexity out. To achieve this, you must identify the key tasks — the product’s red routes — and maintain a laser focus on these during development. Few development teams have this focus because their development process is geared towards coding discrete functions and features, not towards the tasks that users carry out. You can’t run a usability test by asking people to play with a feature: instead, you need to ask them to carry out tasks that require the use of several features and functions. By asking the team to identify the key tasks, you’ll go someway to turning the development process on its head and making it more user centred.

You’ll flush out the stakeholders

Once people hear that you’re running a usability test, you’ll initially think you’re the most popular person in the organisation. Messages will arrive from people you’ve never heard of, often with ‘VP’ in their job title. They’ll want to know why you are doing this work, who authorised it, why you are speaking to customers, what the output will be and why you can’t just speak to the Head of Sales instead (after all, his team speak with customers every day). These people may also provide a whole host of reasons why you shouldn’t go ahead with your test. You’ll find this uncomfortable but it’s an obstacle you need to climb if you’re ever to do any kind of user research in the future.

You’ll establish if there’s an appetite for user research in your organisation

If you can’t get permission to run a usability test, you won’t get permission to do more in-depth user research, like field visits. But once you get the green light, it will make it much easier for you to carry out other, more ‘logical’ user research activities. A usability test will also give you insight into your development team’s appetite for user research: it’s easier to get your team to observe a usability test (even if it’s just watching a highlights video) than it is to get them to come out on a field visit. If they’re not even willing to watch a usability test video, you’ve got a lot of convincing to do. It might even be time to start looking for a user research job elsewhere.

In reading over this article, I’m aware that I sound a bit like a man with a hammer, seeing a world full of nails (when a screwdriver might be a better tool). So in case I’ve not made myself clear, I don’t think that usability tests should be your only user research activity. However, if you’re in an organisation that’s taking it’s first steps in user experience, then a usability test is almost always the best user research activity to try first.

 

David Travis, April 2016