As a Senior Human Factors Consultant I moderate formative and summative evaluations worldwide and it never ceases to amaze me how differently I have to ask questions depending on where I am or who I speak to. Wherever you are there are a number of influences which affect the way you communicate with others. Whether this is age, culture, social status, language or outside influences, the key is being able to learn and adapt your approach to the individual user(s) you are evaluating to find the best way to communicate. I am reminded constantly of this wherever I go!
Instead of asking a question and getting feedback or opinions directly, I had to think of a way to get feedback and opinions without people realising! How can you do this?
Project techniques are a great way to tap into an individual’s subconscious. By getting a user to role play or imagine a scenario and then getting them to finish the scenario, you allow the user to project their feelings and thoughts into someone or something else. So for example a question in this scenario could be: “If your laptop was a fully interactive futuristic robot what would this robot’s function be?” Once the user thinks about their response to this, you can then probe them to explain their answer. For example if the user said “The robot would have the ability to instantly find someone in the building” you could start to analyse this as a possible need that the user wants their laptop to have instant messaging capabilities or in-built GPS synced to other peoples smart-phones or smart-watches in the building. Similarly a response like “The robot would be able to time travel” could mean that the user wants a better way to search their history on their laptop or pull up information easier without trawling through hundreds and hundreds of tedious long winded menus. Once these types of findings are elicited then you can probe and check your understanding.
Sometimes it’s not the question but the method by which you ask the question, or by the adaptation of an existing question to suit the needs of the user. A lot can be said about using scenarios to get the user in the frame of mind of the question you are about to ask them. If done correctly a scenario can be really powerful in not only retrieving an answer, but also in making a user feel at ease and more comfortable in an evaluation so that they can be more open and honest. However, if done incorrectly and your scenario is too long or has too much detail then you can leave a user feeling very confused, frustrated, annoyed and unable to answer you. The key here is to remember that people have a tendency to listen a lot at the start, a little in the middle and a lot at the end of a question or a scenario.
So, if you want feedback on how intuitive a new tap for a shower is and you give a scenario like: “I want you to imagine that you want to take a shower. You have recently cleaned the bathroom and all of your toiletries are laid out as you like them. Please can you tell me what you would do next?” This may sound like a strange scenario to ask in the first place but the point is that quite often it is tempting to fill a task scenario with too much detail in an attempt to help frame the users mind set and embed them in the situation, that we add pieces of information that only distracts the user and is not actually relevant. For example by including details about the bathroom having been cleaned the user might think that they have to talk about cleaning, or by mentioning toiletries the user might think that the next steps would be to use those toiletries. Instead try “I want you to imagine you are about to take a shower. You are standing in the shower cubicle, please can you show me how you would interact with the shower?” The key here is to self-evaluate your use scenario. Try to answer it in as many ways as possible to see how a user could misinterpret the scenario. By doing this it allows you to generate clearer more effective task scenarios which can also be applied to questions.