medium bookmark / Raindrop.io |

Here at WhatUsersDo we’re all about bringing the maximum amount of learning to people at any stage of their UX development. Hence we’re bringing you this series of weekly UX beginner’s guides, aimed to help all the newbies of the UX testing world. This is also why we’ve started using words like hence to make us sound learned.

This week: card sorting!

How does card sorting work? What are the benefits? What do you need to be aware of? How can it help improve the usability of your website? What’s the difference between open, closed and hybrid card sorting? WHAT DOES IT DO, FOR THE LOVE OF GOD TELL ME IN PLAIN ENGLISH PLEASE???

Well, when you’ve calmed down a little bit, I’ll answer your questions and then perhaps we can go outside for some fresh air.

What is card sorting?

Card sorting is a test you can run to improve the navigation of your website, and like all the best methods of UX testing it puts the user at the centre of your observations.

In card sorting, participants are presented with a list of items (for example, all the products featured in an online supermarket) and asked to group them in a way that makes the most logical sense to them.

Depending on the type of card sort, participants can also choose names for the groups they’ve put together, forming the potential categories and subcategories of a website.

All of this will hopefully create an easier, more logical way of navigating your site.

Example of an online card sort from Optimal Workshop. You can also use physical cards.

What does card sorting have to do with Information Architecture (IA)?

You’ll probably hear the phrase Information Architecture (IA) a lot when you begin testing how people navigate your site, and this simply refers to the way content is presented and accessed from any given page – whether that’s through menus, breadcrumbs, categories, links… whatever takes you from one page to another.

This is how our UX researcher Hazel Ho refers to IA in her post on navigation UX best practice:

Information Architecture typically focuses on…

• Structure: the way information is laid out i.e. people should be able to predict where to find what they’re looking for
• Organisation – grouping information in a way that makes sense to people
• Labels – ensuring elements are appropriately titled so people can find information

A good IA can help people on your site understand where they are, what’s around, and what to expect.

One of the many ways you can help visitors find their bearings on any page of your website is by offering a clear, comprehensive and logical means of navigation.

This is where card sorting comes in.

What are the benefits of card sorting?

As mentioned earlier, card sorting is a test you can run to validate the effectiveness of your site’s organisation, structuring and labelling.

It will also help you decide how to label your categories and navigation, arrange the subcategories underneath parent categories, decide what needs to go on a homepage and figure out where users are getting lost or confused.

The ultimate benefit is that you’ll be building and improving your navigation by observing how real users will navigate your site and its information architecture, rather than just guessing yourself.

Just because you assume that people will find the ’Fruit and Vegetables’ subcategory under a ‘Fresh’ category, it doesn’t mean people won’t also go looking under ‘Frozen’.

Card sorting is also simple to arrange, the materials needed are cheap to produce and it doesn’t take too long.

What are the disadvantages of card sorting?

According to Boxes and Arrows, there are a few things to be wary of when running card sorting:

• Naturally the results can vary wildly from person to person, or group to group. Much like any user testing, there’s no guarantee of consistency. But it can give you a solid foundation to build your IA.
• Although the test itself can be quick, the analysis can be time consuming – particularly if there is little consistency between participants.
• All the participants have are the surface details of the content (a title or headline for a webpage) they won’t necessarily know what the content is about or understand any nuance between cards – especially if its a niche subject. Bear this in mind when looking for participants.

How do I run a card sorting test?

The beauty of card sorting is that it’s pretty low tech. In fact you only really need some post-it notes, a wall or a large table and some test participants.

First of all, decide what type of card sort you want to run. There are three possible types of card sorting…

What’s the difference between open, closed and hybrid card sorting?

Open card sort

This is the most flexible option. Participants are asked to group cards into categories that makes sense to them, and then they label each category in a way that they feel accurately describes the content.

Closed card sort

Here participants sort cards into category groups that you’ve already labelled and defined. For example, this is handy if you’re launching a new page for ‘Fitbits’ and you’re not sure whether to put it under a parent category ‘Technology’ or ‘Sports and Leisure’.

Hybrid card sort

As you’d expect this is a mixture of open and closed. Participants can sort cards into categories you’ve already defined and then create their own categories if they think your categories are a bit rubbish.

Then you have to decide on your technique…

Techniques for card sorting

Usability.gov provides the following advice on the different means of card sorting…

One on one

These are in-person sessions with an observer. Users are encouraged to express their thoughts out-loud to give a clear picture as to why they are making their decisions. This can be done with physical cards or online software, and you will need a person facilitating/observing.

Group

These tests have their participants sort the cards out as a group. The group can be briefed at the beginning and at the end of the session, but they are largely left alone for the duration.

This method can lead to quick decisions, but may also be prone to the pitfalls of group dynamics – i.e. the loudest opinion holds the most sway.

Remote

These are online based sessions that require participants to work independently from their own computer. These sessions can be done using open or closed methods.

According to Usability.com, there are several software programs that exist to help you with large-scale remote card-sorting studies, which can also help to analyse the data for you.

How to prepare your cards

You’ll want to create between 30 – 60 cards. Fewer cards may not generate enough groupings, more cards will probably lead to fatigue/boredom.

You then have to decide on the content of your cards.

Optimal Workshop has come up with a list of ways you can come up with ideas for content:

• Brainstorm the different information you want to include on your website
• Look at your existing sitemap or product inventory
• Complete a content audit on your website and select the most relevant items or article titles
• Get a list from stakeholders telling you exactly what they want to see
• Study your intended customers to find out the kinds of information they’d find useful
• Research what your competitors have on their websites

Once you have all your ideas written down on a lovely spreadsheet, you can refine the possibilities until you’re left with only the most relevant cards.

Example from Optimal Workshop.

And finally, you’ll have to find some participants…

How to find the right participants

Whether you’re running one-to-one or group sessions, it’s a good idea to recruit participants who will give you real-world data. As Optimal Worshop states, “one of the goals of card sorting is to get inside the minds of the people you design for” so therefore it makes sense to broadly recruit from your existing demographics.

Things to bear in mind when working with participants

• If people aren’t receiving an incentive or are not obligated to participate, you’ll need to cast your net wider.
• Aim for a minimum of 20 participants for a card sort, many more (upwards of 50) if you have larger menu structures.
• If you’re using physical cards for a group sort, make sure you have enough room for people to spread out the cards and not crash into one another.
• Let participants know how long the sort should take, so it will help them gauge how much time and effort they should put in.
• For goodness sake, if you’re handwriting the cards, make sure your writing is legible. Better yet, print them out.

How does WhatUsersDo run card sorting tests?

I talked to our founder Lee Duddell about how WhatUsersDo runs its card sorting for its clients, this is what he had to say…

Uniquely we blend quantitative and qualitative data into our online card sorting, using tools such as the aforementioned Optimal Workshop. Users on our panel organise pre-defined cards while crucially speaking their thoughts as they do so.

Where there is commonality, the quantitative data can support your decisions. When you need to find out the reasons why people haven’t grouped stuff together, you can then listen to the videos we produce to find out why.

Here’s a clip of a participant using WhatUsersDo to run an independent, unmoderated card sort on a banking website…

More pro tips for card sorting

Here’s a collection of pro tips that you should remember when running card sorting sessions, rounded off with a a few ‘pro-pro tips’ from Lee Duddell.

• It will help to number the cards individually on the corner, as you’ll need this for later analysis.
• Remember that you might have to nudge your participants to think out loud.
• If you’re running concurrent group or independent sessions, remember you’ll need to provide enough sets of cards.
• Just running quantitive test will only get you so far, you need qualitative data (the spoken out-loud observations of the participants) to fill in any gaps of understanding.
• Run card sorts in local languages. Qualitative studies in open card sorts that take into account the nuances of local language will be valuable in ensuring your navigation is understood in international versions of your site.
• It’s worth remembering that top level menus communicate to people what they can do on a site. They form a major part of a new visitors’ first impression, therefore it’s worth running tests on these.
• Menus are really hard to A/B test. It’s difficult to drive decent volumes of traffic to split test different configurations of IA, therefore the qualitative approach of card sorting is necessary for menus.
• You will need to run a tree-test to validate your analysis. More on tree tests next week!

How to analyse your card sort results

If you’ve used card-sorting software, this should generate a report for you. However if you’ve used physical cards – you’ll need to analyse the results yourself.

Usability.gov gives the following advice, which I will distill here:

• After every sort remember to photograph the layout of the cards.
• Write down the names the participant gave to each grouping (if it’s an open sort) and the corresponding numbers of the cards included under that name.
  • Remember to reshuffle the cards for the next sort.
• Write down qualitative information in the form of user comments.
• Write down quantitative information based on: which cards appeared together most often, how often cards appeared in specific categories.
• It may also be worth noting how much time it took to complete the sort.

You can use this data to find commonalities between the card sorts, or if you have a larger number of cards and sessions you could use an Excel spreadsheet to reveal the relationships and patterns between the cards.

Example from Optimal Workshop

What should you look out for in your card sort results?

Optimal Workshop gives some pointers to what you should be looking out for in your results.

With open and hybrid card sorting, you’ll want to answer the following questions:

• What logic do participants follow in the categories they’ve created?
• What cards do people put together in the same group all the time?
• What cards are never put together, and are thus considered conceptually different by all participants?
• What kinds of labels do people suggest for representing your information?

For closed card sorting, you’ll want to look at:

• How many participants sorted the same cards into each category
• The cards with the highest agreement on where they belong, and the cards with the lowest
• Which categories meant different things to different people (ie. if every card was sorted into one category at least once, then it’s obviously ambiguous…)
• What were the most popular groups?

After you’ve analysed the data from card sorting and answered the above questions, you should hopefully have enough information to begin structuring (or restructuring) your site’s information architecture.

Main image by Fahan Siddicq.