Using surveys for product development 10 May, 2006
My experience with Mori got me thinking about how I use surveys to gather user data. Questionnaires are a great tool for providing a range of information. They can also be a useful adjunct to activities using small samples, such as observations or usability evaluations, where clients may be sceptical of the wider applicability of the findings.
I like to use questionnaires to provide context at the beginning of a project. The results give you a baseline and you can re-use many of the questions during usability evaluations to give continuity to your data. For ongoing product development, I’ve found that certain types of questions provide the most useful results.
Likes and dislikes
What do you like most about using <product>?
What do you dislike most about using <product>?
These questions can give you some very visceral responses. You’ll get tactical information about particular features - ‘I hate that it doesn’t remember my information from the Details screen’ - and strategic replies that reveal more about underlying motivation and usage - ‘It lets me keep on top of my appointments and manage my time’.
Product use
How often do you do the following activities with <product>?
| Daily | Weekly | Monthly | Rarely | Never | |
| Create a new page | |||||
| Edit an existing page | |||||
| etc. |
These responses can help you determine where to focus your development effort. If something’s used rarely or never then perhaps it’s time to think about dropping it. Or maybe, if it’s important functionality, you might need some qualitative research to find out why it’s not being used.
Product feelings
For each statement please indicate to the level to which you agree or disagree.
| Strongly agree | Tend to agree | Tend to disagree | Strongly disagree | |
| I thought that <product> was slow | ||||
| Learning how to use <product> was easy | ||||
| <Product> responded quickly to my actions |
Here, you’ll get an understanding of how people see your product and the match between your view of it and your customers’. Some standard statements you might want to include are ease of use and ease of learning. If there are key features or benefits that you’re using to sell your offering, put those in here too and see how they fare.
Whatever questions you settle on, shorter is definitely better for surveys. (This is especially true on the web, where random sampling means you’ll usually be interrupting someone engaged in a totally different task.) People are more likely to answer a short questionnaire (unless you’re providing an incentive), but there’s also the amount of data you generate to consider. You’re going to have to analyse all the responses from that 100 question monster. So shorter means you can discover the trends more quickly and act on them sooner.
If that’s piqued your interest, the following books have loads more advice:
Judith Bell’s Doing Your Research Project and
Martyn Denscombe’s The Good Research Guide.
And for specifics about surveys in product development try:
Kathy Baxter and Catherine Courage’s Understanding Your Users and
Mike Kuniavsky’s Observing the User Experience.