- A blog about
- customer experience
- design
- ideas
- me
- media
- methods
- miscellany
- reviews
- usability
Getting People to Talk: An Ethnography & Interviewing Primer is a wonderful video made by students at the Illinois Institute of Design. It’s also a great example of a Participatory Video approach as one of the creators, Gabriel Biller, notes on his blog:
… we hope that this will be just the first of many videos produced by ID students to help teach others about techniques, methods, and other aspects of design through the powerful and entertaining medium of video.
I won’t try and summarise the film here other than to say that it does an excellent job of demystifying ethnography and showing the benefits of getting in to the field.
Radio 4’s Political Animal recently featured a fantastic rant from Chris Addison on the banality of television news.
There is a tremendous lack of political engagement that we now feel in our county. And conventional wisdom has it that this is to be laid at the door of politicians. We’re not attracted to politicians; we find them cavilling and tedious and lying and so we don’t like politics. But I think that view hugely lets off the hook a whole bunch of other people who are massively responsible, and that is the people who make and broadcast television news. Because television news has become so insulting to the intelligence that if you are not shouting at the television news I believe you should medically be declared dead! I genuinely think an official way of telling if you are alive or not should be turning the news on….
They just think we’re idiots… Tonight with Trevor Mcdonald … it’s got incedental music on it! You’re a current affairs programme! Incedental music is designed to tell us how to feel. So there’ll be a voiceover saying ‘… And at the age of twelve James contracted lukemia …’ And underneath there’ll be Albenoni going ‘nah nah nuh nere’. You think, ‘Oh that’s lucky because for a minute there I was wondering which way to go on little Jim’s lukemia; turns out it’s a bad thing.’
The news editors, on one hand they think we’re idiots and on the other hand they seem to be desperate to know what we think about things. Stop asking us to e-mail the news. I don’t give a shit what George from Grimsby thinks about the emerging Asian economies.
Spot on! The BBC’s coverage of the recently announced crime figures had me bellowing at the box. With absolutely no hint of self knowledge or even any further analysis George Alagiah asked why, when recorded crime has fallen by 10 percent this year, is there such a mismatch between the public’s perception (that it is rising) and the statistical reports? Hmmm. I wonder? If people’s direct experience of crime is that it’s falling where on earth could they be getting the idea that it’s on the increase? The only place where the BBC even alludes to this is in the third paragraph of an article on their website:
While we in the national media may have been highlighting brutal knife slayings, the reality for most parts of England and Wales is completely different.
That’s the real story; sloppy, sensationalist journalism, but for some reason the mainstream media aren’t shouting about it.
A few years ago I read an article that changed the way I do usability testing. In the June 2004 Gotoreport Erik Burns introduced café usability testing: recruiting and running usability tests with participants in local cafés. This was a revelation to me. Even though I was using ‘discount’ methods and didn’t maintain a formal lab, Erik’s method of selecting participants in situ offered the opportunity to streamline the whole recruitment process. I could evaluate designs more quickly and economically, and pass the savings on to colleagues and clients. I ran my first café usability session in 2005 and I’ve been hooked ever since.
If café usability’s new to you and it sounds like a useful technique, then here are a few tips to get you started and hopefully avoid some of the common pitfalls.
You can run café usability sessions anywhere. You don’t need to restrict yourself to cafés. Conferences, trade shows, events, museums, canteens, showrooms, student unions, user groups; wherever you think you’ll find people who match your target audience.
The bigger your recruitment sign, the better. In café evaluations you’re recruiting participants on the spot. If you’re on your own, your main tool is usually a sign offering an incentive. A4 desk signs are OK in public spaces where you need something inviting yet unobtrusive, but if you really want people’s attention you can’t beat a big poster. Position yourself next to a wall and stick the poster above your table (provided you’ve got permission, of course). Your sign does all the hard work and you can focus on the evaluations.
Get creative with your incentives. A big poster means you’ll get people’s attention, so now you need to concentrate on how you’re going to entice them to participate: the incentives. There’s less rigmarole for participants in café evaluations than in standard lab tests. You can be more adventurous and, um, budget conscious, in the kinds of incentives that you offer. The original article talked about free beer; I’ve offered chocolate, Gmail accounts, ice lollies, champagne and vouchers as well as the old workhorse hard cash. People tend to respond well to something a bit out of the ordinary, so go wild!
Position yourself where your participants are. This may seem obvious because you’re out in the field already, but your location in the chosen venue can really affect how many people you recruit. If possible try to visit the site beforehand to get a feel for busy times and places. Flexibility is the key; if you’re not seeing enough participants then it may be time to move.
If you’ve got help, recruit people away from your base. Take turns to go around the café/conference/canteen/hall/wherever and ask people if they’ve got a bit of spare time. Print some cards or simple paper leaflets with your company name, where you’re located in the venue and an outline of what you’re doing. Hand these out to people as you go around. At conferences, for example, there are often lulls early in the morning and late in the afternoon. Visiting other vendor’s stands and telling them about your evaluations can provide extra participants during these slow times.
Run your evaluations in public. I’ve seen some correspondence lately suggesting that you recruit people in public areas, but conduct the sessions somewhere private. Informality is the key to this technique. Running your sessions in public keeps it that way. Once you’ve withdrawn to another room you may as well be back in the lab. Nevermind that now you’ve got to manage participants in a separate room as well as maintaining your recruitment setup.
Bring spares of everything. In the lab all your supplies are usually to hand. In the field you’re dependent of what you (and any colleagues) can carry. You’d be surprised how easily simple things like mislaying pens or running out of paper can derail a session. So extra pens and paper are a must, but I’d also bring lots of consent and recording forms, copies of any paper prototypes (parts can easily get lost or damaged) and spare batteries for your laptop. And that’s just for starters. I’m sure there are lots of other things that you can think of for your specific situation.
Listen first, then ask. Café evaluations are conducted away from the lab, so this is a great opportunity to let your users lead and observe their behaviour. Try Mark Hurst’s listening lab approach rather than using pre-defined tasks. Talk to participants to discover what they would normally do on a website or application like yours. Note their answers and then ask them to try some of these tasks with whatever artefacts you’ve got (full site on a laptop, prototypes etc.). If there are specific areas that you want to test, throw in a few exercises of your own, but leave these until you’ve had a chance to see the participant’s usual behaviour.
Allow for more time, but don’t count on it. I usually ask for 20 minutes of people’s time, but the informality of cafe evaluations is infectious. Once participants have started they’ll often give you far longer. Have a few standby tasks or questions ready to make the most of each session. As with all research though, respect for the participants is paramount and that extends to their time. If you’ve asked for 20 minutes then that’s what you should aim for. (You’ll be surprised at how much you can achieve in just that short period.)
The 10th tip? Well that’s up to you. If you’re already running café usability sessions then I’d love to hear your favourite piece of advice. How do you do it? What do you do differently? If you’re just getting started with café evaluations then let me know how you get on. What did I overlook and what would you add? I’m sure that you’ll find the informality and freedom of café usability as stimulating as I do, even without the coffee.
This post originally appeared, in a slightly modified form, as an article in issue 75 of Interfaces, the magazine of the BCS Interaction group.
Fantastic to see Mark Cavendish become the first Briton to win three stages of the Tour de France. (He even got a mention in the headlines on PM, a program that generally shuns sport in favour of lightweight political coverage.) Cav’s exploits have lifted a race that was always going to have difficulty competing with the drama and spectacle of last years event, but there’s one thing that stays the same no matter how the competition’s faring – you always ride that little bit faster when it’s on.
Dan Saffer asks about new techniques to try in 2007. (We’re closer to 2008 now, but I started this in response to Dan’s post so I thought I ought to finish it off before we hit New Year again.) Participatory Video (PV) has been around for a long time, but with YouTube making national news and video mobiles well established, a cut-down version could provide a useful addition to the toolbox of methods available to researchers and designers.
PV creates a story told by users in their own way about issues that matter to them. In the traditional version, participants film and share short videos. The researcher acts as a facilitator providing training to use the video equipment; a series of games to introduce specialist concepts like storyboarding; and help to identify issues for study. When done well, it presents the ‘inside view’ in a lively way that is accessible to people at all levels. It promotes the skills of filmmaking and storytelling and delivers outcomes that can be used as tools for education. This isn’t like a diary study, capturing things exactly as they happen when they happen. It’s about people telling a story, distilling the essence of an issue that matters to them. Remember a group project you did at school or college, then add video and you’re getting there.
A streamlined version could see some of the facilitation done online or through pre-prepared packs, with people recording clips on mobiles then exchanging and responding to them using services like Google Video or Viddler. Imagine workers at a car production plant making a film about their day, or a group of diabetics sharing the story of their condition.
If this sounds interesting, why not check out Insight UK’s fantastic ‘Insights into Participatory Video: A Handbook for the Field‘ (you can download a free PDF version of the book from their website; you just need to register a few details). It’s an excellent guide to using Participatory Video and a great example of an accessible and practitioner-focussed resource. Even if you don’t think there’s much mileage in PV I’d still suggest having a look at the handbook. It really is excellent. (I’m not in any way associated with Insight UK, I just think they’re doing good work.) And anyway, it looks like Participatory Video is already happening on sites like YouTube. The question is whether we as designers want to embrace it.
If you’ve read more than just this post (hurrah for The Subscriber), you’ll know that I believe a little research can go a long way. I also believe in democratising research, especially in team settings, so that everyone can learn. This story in the Daily Mail shows that anyone can get into the field and start learning about their customers.
Abramovich’s wife goes undercover as she learns how to run a hotel chain
Irina recently paid a number of visits to the chic Mayfair establishment Brown’s…, not as a guest - but as an undercover trainee. The elegant Russian has embarked on secret ‘work experience’ at the five-star hotel.
You don’t have to be a trained user researcher or a billionairess to live in your user’s shoes (although either helps), you just have to be willing to get out there and meet with them; to be open to what they’ve got to teach you.
So what are you waiting for?
England’s recent friendly against Brazil gave me my first glimpse of the new Wembley stadium. For an architectural structure that’s created so much controversy, taken so long to build and cost so much it seems a bit, um, underwhelming.
The ‘triumphal’ arch is impressive from a distance, but there’s no element of surprise or delight to the stadium. No emotional connection.
Contrast that with Munich’s Allianz Arena.
The outer shell changes colour depending on which of the local teams is playing. Red for Bayern Munich, Blue for 1860 Munich (with white for derbies and Internationals). Amazing and guaranteed to ignite passions. Imagine being a home supporter and seeing your team’s colours enveloping your stadium. The simplest, yet most visceral connection for any football fan. Suddenly, all of the triumph seems to have disappeared from that arch.
Talking to a friend who’d visited Wembley a few weeks ago it seems that the architects got some things right. He marvelled at the number of toilets (it’s always details); 2,618 of them according to the official Wembley site. A lesson learned from the limited facilities at the old stadium which lead to some, er, interesting workarounds from spectators. At half-time there were usually three queues of people waiting to relieve themselves: one for the cubicles, one for the urinals and one for the sinks. Still, I suppose it makes for easy access to the taps afterwards.
Photography from sismastery, rp72, gonzales2010 and probek on flickr.
The paradox of interface design is that when it’s done well you shouldn’t notice it. Good interfaces seem natural because everything you want is to hand. (A principle described as what-you-see-is-what-you-need by software interface heavyweights Constantine & Lockwood.) It means that you can get on with what you’re trying to do rather than worrying about how to do it. When you don’t see what you need breakdowns happen; flow is broken and suddenly the interface is the problem.
Consider this screen from Amazon’s Marketplace service. The idea is to provide feedback on your purchase to help drive their reputation system. And it works well, up to a point. The designers even remind you of the information you need to complete the task in the ‘Questions to consider’ panel, but then don’t provide an interface to all of it! Look at the second question. It’s probably one of the most important things in any transaction: ‘Did the seller accurately describe the items?’. Yet, despite having this information in their database, Amazon expect you to to recall the original description from memory.
This constantly frustrates me because it makes it difficult to give accurate feedback; the cornerstone of any reputation system.
Here’s my revision. Simply adding the original description (by exposing a field in a database) gives me the information I need to properly rate the book purchase.
To know what people need in their software interfaces you have to appreciate how the software is being used; what the people using it are trying to achieve; and the information they need to get that job done. And having made the effort to understand what’s to be done you need to make sure that your application supports it.
There’s a useful new twist on an old technique over at frog’s Design Mind. In their digital diary study participants used voicemail, e-mail and digital cameras to record their behaviour. These insights were sent daily by e-mail and free phone numbers to researchers who followed-up through the same digital technologies. Frog believe that this rapid feedback created a conversational dialogue and a more responsive study. According to them, the technique is:
- Efficient (digital methodology and remote facilitation require no travel)
- Rapid (digital data collection accelerates response)
- Scalable (process allows for parallel investigations in many countries)
- Dynamic (immediate communication permits follow-up questions)
I can certainly see the advantages, and use of the technology would fit perfectly with certain groups like teenagers. But whilst Frog’s view is that the application of ‘digital’ facilitated the more conversational/ethnographic approach …
Feedback fostered a dialogue between researchers and subjects that mirrored true ethnographic interviews…. Traditional diaries allow for no interaction between researchers and subjects…. Data is restricted in one direction.
… I would argue that there’s no reason why paper-based studies can’t include a dialogue with the participants.
The paper form that I created for my student technology study. Participants reported and categorised their activities throughout the day. The form was based on the work of John Reiman.
I was in regular communication with participants when I used diaries to investigate the way students integrate technology and studying into their lives. I met with the students to brief them about the study and then followed-up each week with a face-to-face or telephone interview. Participants were given disposable cameras, alongside paper forms, to help document their behaviour and surroundings (this was before camera-phones or cheap digital cameras had become widely available). E-mails and texts helped to arrange formal feedback sessions and provide informal support between meetings. Not quite the information deluge that the Frog Design team solicited, but enough to keep this researcher busy in-between.
Digital diaries are definitely an exciting and agile method, but with many technology-driven approaches the technology itself can often become a barrier. I think there’s still a lot of scope for mixing the digital and traditional.