One of the most controversial debates on migration is around the links between aid, policy programming, development and migration. But for researchers to be able to draw meaningful conclusions about these links, data collection needs to reflect the entire population, not just those on official population registers.

And this is not just a technical issue: choices made about sampling shape the findings and policy implications one can draw.

From ensuring samples are representative to population data being unavailable or out of date, sampling is hard, and there are trade-offs to make.

This is particularly problematic in areas with informal settlements and high rates of population change, where migrants and other vulnerable populations might not even be included in official population lists.

The good news is, you can still build a valuable, insightful and reliable survey sample by thinking about these trade-offs carefully, as we did while designing our recent MIGNEX survey.

MIGNEX – Aligning Migration Management and the Migration–Development Nexus – is a five-year research project with the core ambition of creating new knowledge on migration, development and policy. The survey is a key component, and when it resumes next week, we will be interviewing 12,500 young adults across 25 research areas whose lives may be touched by the dynamics of migration and development.

Here are five key lessons we have learnt that can help you overcome your sampling woes and build a better understanding of migration and development, particularly across Global South countries.

1. Be realistic

It goes without saying that sampling processes need to be adapted to the context. In MIGNEX we want to collect comparable data across ten countries, which means working with the lowest common denominator.

For example, even in the countries where up-to-date population data exists and a more sophisticated sampling strategy could be possible, we instead designed and implemented a clear, simple and easily replicable process that could be applied across all contexts.

2. Innovate

Increasingly, researchers have been making use of advances in Geographic Information System (GIS) technology (PDF) to draw representative samples.

However, our survey team is split across several institutions, most with limited experience in GIS methods. So, we developed a simpler approach that draws on GIS, which can be done digitally or by hand.

To overcome the lack of population data for our sampling frame, we use satellite maps to identify smaller areas (clusters) within each village, town or city. We then count the buildings within each cluster, making transparent assumptions about how many households live within each building.

For example, the size of a shadow can be an indication of whether one or more households live in a building. This is complemented with a verification exercise, where we compare our virtual household estimates to the reality on the ground.

3. Try it out before locking yourself in

Piloting survey questionnaires is standard – but piloting sampling approaches? Less so. We were not sure whether our sampling approach was going to work, so we gave it a try in São Vicente, Cabo Verde in February 2020. To test how accurate our desk-based household estimates were, we counted the number of households for two small areas.

In essence, this was a simple exercise of walking along the streets of each selected area. We looked at and counted the number of households that appeared to live in each building, making a running tally of households in the area.

The results were promising, although they also revealed some problematic assumptions we had made. For example, we overestimated the number of households that live in multi-household buildings. As a result, we revised our sampling strategy to include an explicit step of verifying assumptions with locals familiar with the area, as well as on the ground-verification exercises mentioned previously.

4. Do simple, well

The gold standard for selecting respondents is to randomly draw them from the sampling frame in advance of data collection.

We can’t do that, both because our sampling frames are too basic and because we are interested in a particular population sub-group of 18- to 39-year-olds. We focused on this particular group because their lives are most likely to have been touched by migration in one way or another.

To overcome this, we use a random walk for sampling respondents. This methodology involves systematically walking through neighbourhoods to sample households, covering all types of residential dwellings, including informal settlements.

Random walks are often criticised for not drawing representative samples, although some new approaches (subscription required) have addressed these issues by sending enumerators on ‘planned random walks’.

Here, we decided to ‘do simple, well’. We set out clear, detailed and replicable guidelines and created training materials covering all aspects, from starting points to navigating dead ends, to ensure random walks were done consistently.

This approach allows us to capture all people in our population group of interest, regardless of their immigration or residential status or the type of dwelling they live in.

5. Don’t rush it

We have spent the past year designing the MIGNEX survey, weighing up potential benefits and disadvantages of different approaches. Having the luxury of this time meant that we were able to take piloting seriously and try different approaches, before making final decisions.

Given that we want to collect comparable data across 25 different contexts, even deciding upon and implementing a ‘do simple, well’ approach took time. Giving yourself the space to reflect, test and then arrive at the best sampling approach for your research is a critical component and one that should not be underestimated.

Sampling and survey design should not be relegated to the dusty bookshelves of academics: choices made here are critical to being able to make reliable and meaningful contributions to the migration field. Specific step-by-step processes, from sampling to ensuring data quality, are all included in our open-access guidelines published on the MIGNEX website, so we encourage others to learn, utilise and adapt these when designing their own migration surveys. You can also join our upcoming MIGNEX webinar on 8 October, which will take a deep dive into different approaches to survey sampling design.