What happens when usability research is conducted abroad? A team from PDD have been working closely with international manufacturing organisations to develop medical devices for the Chinese inpatient market. This means that usability research has been conducted in Chinese and reported in English. This blog explores the benefit that usability research provides when implemented across international teams and how to overcome issues that may be encountered along the way. It amalgamates our insights taken from a number of usability projects conducted in China.
Usability research in China
Usability encompasses effectiveness, efficiency, ease of learning and user satisfaction. In many ways conducting usability research in China is no different from conducting it elsewhere in that there is a need to consider what techniques will deliver the best insight. There is a balance between collecting data that is free from influence and interpretation (objective accounts) and those based on opinion (subjective accounts). On one hand the assessment of usability needs to be based on the clearest possible evidence, on the other hand the experiences and preferences of users (as verbalised during testing) can be subjective, impacted by cultural norms and subject to multiple interpretations during translation.
It is important to involve a local team that knows about the product (and product design)
We all know about the potential for meaning to become lost in translation, but this is not an issue if the right approach is applied. For example having a translator that is part of the project team and knows the product can help them provide a representative account. This also applies to the moderator who needs to be in a position to understand and explain complex medical concepts in English and Chinese. It helps if they are familiar with the design and engineering process and form part of a broader team working on a medical device project. A lot hinges on the skills of the moderator and translator so getting the right team is critical. It can also help to use multiple independent note takers, agree on parts of an interview script that are difficult to translate ahead of time, and use pictures and references to support communication.
Getting these things right is well worth the effort
China has a great tradition of educational and scholarly work. Running usability tests in Chinese cities has been a rewarding experience (experiences from rural locations will be reported in a future article ). Participants understand the need, are constructive in their approach and respectful towards the process (turning up on time, giving substantive feedback and being honest). It is worth confronting some of the challenges in order to benefit from logical, well-reasoned and considered feedback. The best way to do this is to work closely with a local team.
Benefits of usability testing
When things go right, the benefits of usability testing are substantial and can be shared across multiple organisations. For example in China it was not unusual for tests to be observed by the maximum number of people who could fit into the market research facility, representing a broad community of practice (e.g. design, engineering, human factors and ergonomics, marketing, regulatory approval, risk management and training). In this case a degree of sensitisation occurs both to the needs of the user and the perspectives of others (who might think differently about the results of the testing). Understanding "real world use" might reveal some surprises for both the development team and those who are responsible for marketing a product.
Conducting usability research helps a manufacturer learn about how their products are really being used. In this way, usability research:
• Makes assumptions explicit and highlights conflicts in understanding.
• Helps progress product development and supports joint reflection.
• Allows for a process of adjustment, reinterpretation and joint negotiation.
User research therefore becomes a tool for learning, engagement, reflection and communication, rather than a process of overcoming regulatory hurdles. In this way the activity approximates a process of action research - a reflective process that is undertaken by those within a community of interest.
At a purely practical level, working in China can be full of surprises such as power outages, technical issues and last minute changes to schedule. The cities are vast, hectic places. A local team helps to keep things running smoothly.
It is easy to overlook one of the fundamental benefits of conducting this type of research - namely the potential to bring international teams together. Although a plethora of medical device usability guidelines are available, very few consider how international groups work together to design products for a specific market. PDD is well placed to conduct this type of work and provide the local team that is necessary. Our team has over 35 years of experience working on medical device design and usability projects in China involving a range of international clients.