Designing online questionnaires for international organisational perspectives on the COVID-19 pandemic
By Amita Bhakta
We often see surveys in the form of a questionnaire. In the past, we might have been asked to take part in a questionnaire by post, or on the street by those conducting surveys for research. Increasingly, these surveys now come to us directly to our email inboxes in online form, particularly at present as lessons are being learned about the impact of COVID-19 on our work, in a socially distanced way. No matter whether a survey is hosted on a website or printed on paper, there are lots of factors to consider in the design process. In this blog, I reflect on the lessons I have learned in designing a survey over the past two weeks as a consultant for a current project with the International Disability and Development Consortium (IDDC) about the impact of COVID-19 on the work of IDDC member organisations. I have designed and used surveys in the past, such as for desk-based research on menstrual health projects, but working with the IDDC has provided lots of food for thought.
Keep it simple
When setting out on questionnaire design, the key think to bear in mind is to keep it as simple as possible. The first step to keeping it simple is to think about your audience, and the length of the survey. Participants will want to know firstly what the survey is trying to achieve, so that they can consider answer carefully. In this project with the IDDC, we are trying to understand the impact of COVID-19 on organisations who are IDDC members. Writing a paragraph at the beginning of the survey can help to focus the thoughts of the respondent; tell them in simple terms what the survey is about, and what you will do with the data. Adding information on how survey data will be safely managed is reassuring for respondents. This paragraph can also help you to make sure that questions match the objectives of the survey you are outlining, and it is wise to keep referring back to it when writing questions. Keeping it simple and focussed is also about the length. People are not likely to spend too long on a survey, and if they start to see question 21, 22, or 23, you’ve probably gone too far, and the respondent will have lost interest and their answers won’t be as good.
Have a logical order to your questions
I find it is always useful to think about the order of questions from the outset: what do you want to know, but most importantly, how might the respondent be able to think about different issues, and in which order? Asking people about themselves as individuals, their name, their job and where they work can be a good start, because it makes the initial stage about them, and they are likely to then become more interested. After that, you can then broaden the questions to the scope of your project. Again, these questions still need to be tailored to your target audience. Exploring the impact of COVID-19 on organisations working in development through a survey is likely to involve gathering responses from people involved in programmes. For COVID-19 specific surveys, a key aspect to think about is stages of the pandemic. Distinguishing questions chronologically about how programmes operated before the pandemic, during different waves of the pandemic, and in recovery periods helps the reader to reflect on specific time frames and can help to narrate impacts.
Redraft your questions over time, and then pilot them
No one really wants to have to write lots in a survey, but nor can tick box responses tell us everything we need to know. Writing and designing a mixture of both types of questions for a survey requires a lot of thought. Is it a yes or no question? Is there a list of certain categories to use to describe organisations or programmes? Can the respondent’s view be measured through a Likert scale, or do we need them to tell us a bit more detail on some aspects of their work which we do not know about in an open box and cannot provide categories for? There may be lots that some organisations can say about social distancing measures in programme activities, but less to say on the topic for others. The important aspect of this is to know how you are going to analyse the data for a certain question before you begin to write it. Getting this right is quite a process, and on this project with the IDDC, my time so far has been spent working with the team to draft, redraft, and redraft questions again, according to the types of answers we want. It is wise to take time to do this to get it right. A pilot with a small group of colleagues to test whether the questions work before sending it out is a good plan to keep in mind, giving you a chance to address any issues arising with understanding questions and with the functionality of the software itself.
Different countries, different languages
Finally, for a survey of an international scope, it is a good idea to have versions of the survey in different languages. The key reason for this is to be as inclusive as possible and to capture key and important information from participants who might not have English as a first language. Check the languages being spoken by the target audience and look into translation options for an online survey, as some platforms may have options for different languages where as others may not, so choose your platforms wisely! And after that, try, try, and try again with participants, and enjoy learning from others!
Acknowledgements
Thank you to my IDDC colleagues, Angelique Hardy, Dominic Haslam, Ruth Faber, Elaine Green and Ola Abu Al Ghaib at UNDP for the fantastic ongoing discussions in the project which have inspired this blog, and to IDDC for commissioning me to be part of this work.