Usabilatte: 10 tips for running café usability sessions 7 August, 2008
A few years ago I read an article that changed the way I do usability testing. In the June 2004 Gotoreport Erik Burns introduced café usability testing: recruiting and running usability tests with participants in local cafés. This was a revelation to me. Even though I was using ‘discount’ methods and didn’t maintain a formal lab, Erik’s method of selecting participants in situ offered the opportunity to streamline the whole recruitment process. I could evaluate designs more quickly and economically, and pass the savings on to colleagues and clients. I ran my first café usability session in 2005 and I’ve been hooked ever since.
If café usability’s new to you and it sounds like a useful technique, then here are a few tips to get you started and hopefully avoid some of the common pitfalls.
You can run café usability sessions anywhere. You don’t need to restrict yourself to cafés. Conferences, trade shows, events, museums, canteens, showrooms, student unions, user groups; wherever you think you’ll find people who match your target audience.
The bigger your recruitment sign, the better. In café evaluations you’re recruiting participants on the spot. If you’re on your own, your main tool is usually a sign offering an incentive. A4 desk signs are OK in public spaces where you need something inviting yet unobtrusive, but if you really want people’s attention you can’t beat a big poster. Position yourself next to a wall and stick the poster above your table (provided you’ve got permission, of course). Your sign does all the hard work and you can focus on the evaluations.
Get creative with your incentives. A big poster means you’ll get people’s attention, so now you need to concentrate on how you’re going to entice them to participate: the incentives. There’s less rigmarole for participants in café evaluations than in standard lab tests. You can be more adventurous and, um, budget conscious, in the kinds of incentives that you offer. The original article talked about free beer; I’ve offered chocolate, Gmail accounts, ice lollies, champagne and vouchers as well as the old workhorse hard cash. People tend to respond well to something a bit out of the ordinary, so go wild!
Position yourself where your participants are. This may seem obvious because you’re out in the field already, but your location in the chosen venue can really affect how many people you recruit. If possible try to visit the site beforehand to get a feel for busy times and places. Flexibility is the key; if you’re not seeing enough participants then it may be time to move.
If you’ve got help, recruit people away from your base. Take turns to go around the café/conference/canteen/hall/wherever and ask people if they’ve got a bit of spare time. Print some cards or simple paper leaflets with your company name, where you’re located in the venue and an outline of what you’re doing. Hand these out to people as you go around. At conferences, for example, there are often lulls early in the morning and late in the afternoon. Visiting other vendor’s stands and telling them about your evaluations can provide extra participants during these slow times.
Run your evaluations in public. I’ve seen some correspondence lately suggesting that you recruit people in public areas, but conduct the sessions somewhere private. Informality is the key to this technique. Running your sessions in public keeps it that way. Once you’ve withdrawn to another room you may as well be back in the lab. Nevermind that now you’ve got to manage participants in a separate room as well as maintaining your recruitment setup.
Bring spares of everything. In the lab all your supplies are usually to hand. In the field you’re dependent of what you (and any colleagues) can carry. You’d be surprised how easily simple things like mislaying pens or running out of paper can derail a session. So extra pens and paper are a must, but I’d also bring lots of consent and recording forms, copies of any paper prototypes (parts can easily get lost or damaged) and spare batteries for your laptop. And that’s just for starters. I’m sure there are lots of other things that you can think of for your specific situation.
Listen first, then ask. Café evaluations are conducted away from the lab, so this is a great opportunity to let your users lead and observe their behaviour. Try Mark Hurst’s listening lab approach rather than using pre-defined tasks. Talk to participants to discover what they would normally do on a website or application like yours. Note their answers and then ask them to try some of these tasks with whatever artefacts you’ve got (full site on a laptop, prototypes etc.). If there are specific areas that you want to test, throw in a few exercises of your own, but leave these until you’ve had a chance to see the participant’s usual behaviour.
Allow for more time, but don’t count on it. I usually ask for 20 minutes of people’s time, but the informality of cafe evaluations is infectious. Once participants have started they’ll often give you far longer. Have a few standby tasks or questions ready to make the most of each session. As with all research though, respect for the participants is paramount and that extends to their time. If you’ve asked for 20 minutes then that’s what you should aim for. (You’ll be surprised at how much you can achieve in just that short period.)
The 10th tip? Well that’s up to you. If you’re already running café usability sessions then I’d love to hear your favourite piece of advice. How do you do it? What do you do differently? If you’re just getting started with café evaluations then let me know how you get on. What did I overlook and what would you add? I’m sure that you’ll find the informality and freedom of café usability as stimulating as I do, even without the coffee.
This post originally appeared, in a slightly modified form, as an article in issue 75 of Interfaces, the magazine of the BCS Interaction group.
August 7th, 2008 at 8:32 am
Nice post!
One issue I’ve had with cafe based research in the past is screening the participants properly. People often bend the truth to get cash incentives, so it’s important to spend the first 10-15 minutes of the interview asking them extensive screener questions. Are they really in your target user-base? This is pretty fundamental, especially if it’s a niche product.
August 7th, 2008 at 9:41 pm
Hi Harry,
Thanks for the kind words and the very pertinent comment – screening is a great candidate for the 10th tip. I’ve found that being specific in the locations that you choose can do some of that work for you. For example, if you were looking at how people buy gadgets you might pop in to your local Student’s Union where you’ll find lots of people sporting or lusting after the latest gear. If you were evaluating a museum website you could sit in the museum’s restaurant or foyer and talk to people who will have just arrived or who’ve been perusing the exhibits.
If you’re only looking at spending 20 or 30 minutes with each participant you need make sure that you make the most of that time.
January 28th, 2010 at 9:56 pm
Hey Harry, can you give me an example of what would disqualify somebody from testing a tourism website, for example?