Department of Agricultural, Food, and Resource Economics Food Security Group

To flatten the curve and prevent further escalation of the spread of COVID-19, universities and research institutions around the world have suspended all research activities that are not necessary to sustain or protect life. As a result, field research involving interactions with human subjects has come to a standstill. In a time when data and evidence are most needed to make informed policy and programmatic decisions, researchers are unable to do primary data collection. If you are an international development researcher and are feeling helpless about this situation, don’t; thanks to the increasing penetration of affordable mobile phones in developing countries, you can still do some meaningful and timely research while working from home.

Although face-to-face interview is probably the best data collection method, mobile phone surveys are the second best alternative. In times of travel restrictions and lockdowns, they are the only tools available for research. Recognizing some of the advantages of this method, I have used mobile phones for research data collection and also as a tool for information dissemination. In doing so, I’ve learned a few lessons. This entry describes some of the ways researchers can use mobile phones for data collection in developing countries. It highlights the pros and cons[1] of using different modes and some basic operational considerations for designing successful research in the current COVID-19 lockdown scenario and beyond.

Mobile phones for data collection: Different modes, pros and cons

Short message service (SMS), online surveys (web), computer-assisted telephone interviewing (CATI), and interactive voice response (IVR) are some of the common ways to reach respondents to collect research data using mobile phones. These varied ways of using mobile phones can produce data faster and less expensively than face-to-face interview surveys. They also offer advantages like high anonymity and accessibility, especially in times of conflicts, disasters and disease outbreaks like Ebola and COVID-19. They can be designed and deployed in different languages just like traditional face-to-face surveys.

The SMS feature of mobile phones can be used to text questions and answers back and forth. An advantage of this method is that anyone with a cellphone can respond and is thus highly accessible. But this method also has significant limitations. First, it requires respondents to be literate and SMS-savvy. Second, questions and response options together cannot exceed 160 characters, and they are sent one at a time. Research suggests that in SMS surveys, respondents tend to lose interest more quickly than with other modes and thus there is a high rate of survey non-completion. This is in addition to the very high non-response rate for SMS surveys (90-95%).

The SMS feature of mobile phones can be also used to text respondents a link to a web survey they can complete from the browser on their phone. This option gives researchers greater flexibility with question wording. But in most developing countries the penetration of smartphones, although growing, is still only about 20-60%. Therefore, this option takes away the high accessibility advantage of mobile phone surveys. Moreover, SMS-web surveys suffer from the high non-response rate similar to SMS surveys.

Telephone surveys have long been used in developed countries like the U.S. CATI is an extension of this well-established method with enumerators using scripted questionnaires programmed in a computer (or a tablet) and entering the data while the interview is taking place. over the phone. A big advantage of CATI is that it can be done with any type of phone and does not require literacy on the part of respondents. CATI also has higher response rates than SMS and web surveys (about 30-40%) but is more expensive than those methods.

The IVR survey is a pre-recorded automated phone survey in which the respondent responds vocally to questions or enters numerical responses using the keypad. Although IVR surveys are cost effective, they typically have a longer turnaround time from survey deployment to project completion.

Overall, in quick-turnaround projects (not previously planned) that need to reach illiterate populations, CATI is the best choice.

Operational considerations

Is mobile phone survey the best option? With the door for doing face-to-face surveys temporarily closed, more and more researchers are turning towards collecting data through mobile phones. But it is important to recognize that mobile phone surveys are not a perfect substitute for the traditional face-to-face surveys. As indicated in Table 1, mobile phone surveys have several limitations and are suitable only for some types of surveys and in certain conditions. These need to be considered carefully in making the decision to opt for this method.

Table 1. As a data collection tool, what are mobile phones ideally suited for and not suited for?

Which is the best mode? After it is determined that mobile phone survey is the best suitable method for data collection, the next important choice a researcher needs to make is which mode best fits the study’s objectives. In addition to weighing the pros and cons of different modes of doing mobile phone surveys, researchers need to consider which mode provides the best coverage of the target population, the quality and type of contact information on the available sample frame(s), projected response rates (which can be as low as 5-7% for SMS surveys to 30-40% for CATI surveys) and sample size requirements, types of questions, and time and budget constraints.

Who does the survey? An important operational consideration is whether the survey should be done in-house or outsourced? This decision depends on several factors such as available human and budgetary resources, technical capabilities, target population (i.e., sampling frame), and most importantly, access to contact numbers to reach the target population.

Where to get phone numbers? Previous surveys can be a great source of contact numbers. In fact, mobile phone surveys are ideal for follow-up with respondents with whom researchers have already conducted face-to-face interviews.[2] Mobile phone surveys can be implemented by researchers themselves if the plan is to use an existing database of phone numbers from previous surveys and researchers had technical capacity to design and deploy mobile phone surveys.[3] However, in the absence of such database or when surveys are targeted to broadly defined categories of population, the other option is to contract survey firms like Gallup, Pollfish, GeoPoll, etc. who would have access to mobile subscribers if they operate in the country. This option could be relatively more expensive than conducting surveys in-house. But these firms have access to millions of mobile phone subscribers and have more robust technical capabilities to deploy surveys across different modes.

What about data representativeness? Whether you source contact information from a previous survey or use random digit dialing (RDD) method to get a random sample from mobile subscriber’s list, data collected using a mobile phone will suffer from selection bias due to two reasons—1) not everyone in the sampling frame will have mobile phones, and 2) the high non-response rates across modes. If a follow-up phone survey is planned when designing a face-to-face interview survey, these two limitations can be minimized by providing mobile phones to participants if they don’t have it, training all the participants on how to use it, and informing them about future phone surveys at the time of recruitment (see foot note 2). However, if mobile phone surveys were not planned ex ante at the time of face-to-face interviews, then possible ways to address the selection bias in a follow up survey that uses contact numbers from an existing survey would be to adjust the phone survey data using weights to make the data representative of the original sampling frame. A crucial issue in such surveys, however, would be the effect of high non-response rate on the sample size and statistical power. In the case where mobile phone surveys are population based (for a broadly defined population) and the sample is drawn using RDD, the concern of sample size (due to non-response rate) is low, because there is a large pool of mobile phone numbers (millions) to draw the random sample (with replacement). However, in such surveys, the concern would be sample representativeness. One way to address this would be to apply sample weights to reflect the population of interest.[4] 

Who pays for data and airtime? In addition to the time cost, a respondent also incurs monetary costs of airtime and data for participating in mobile phone surveys. To increase the response rate and as a good practice, researchers should cover these monetary costs and compensate the participants for their time in the form of additional airtime as an incentive to complete the survey.

Legal and ethical considerations: Mobile phone surveys need to undergo the same process of Institutional Review Boards as any type of research involving human subjects. However, before deploying mobile phone surveys, it is important to find out if there are any government regulations that make it illegal for researchers or companies to call or send text messages without expressed consent. If researchers plan to use contact information of respondents collected during traditional face-to-face interviews as a sampling frame for future mobile phone surveys, two good-practice tips for the face-to-face surveys are: 1) collect information on the type of phone (basic or smart), and 2) seek and record explicit permission to contact the respondent via cellphone by text or voice call. This second practice will also help increase the response rate on future phone survey.

[1] See excellent blogs on survey methodologies, including mobile phone surveys on the Gallup and Geopoll websites.

[2] Mobile phone survey can be also integrated with face-to-face surveys by providing subjects with a mobile phone to collect data multiple times over the course of a research study. See for e.g., Demombynes et al. (2013), Dillon (2011), and Etang-Ndip et al. (2015).

[3] For CATI surveys, researchers can hire and train enumerators to conduct surveys by phone. For SMS and IVR surveys there are online platforms (like SurveyMonkey, SimpleTexting) or software that can used to design and deploy targeted surveys.

[4] See Lusk (2017) for an example of how this method is used in the context of online surveys on food demand in the U.S.