To respond to the humanitarian challenges associated with large-scale migration and displacement, policy-makers need stronger evidence to design, implement and adapt initiatives that address the needs of children. Our first blog in this series discussed five evidence-based approaches that can better protect children on the move. However, a ‘substantial gap’ between evidence and policy still exists, since migration policy responses related to children on the move respond to a range of factors. Inertia within organisations, funder priorities or political imperatives can all prevent evidence from getting translated into action.
Good quality evidence can play a vital role in driving positive change. ODI research has shown that when the linkages between knowledge, policy and practice are strong, practitioners and agencies can develop approaches that can save lives, reduce poverty and improve people’s quality of life.
With this in mind and based on ODI’s recent UN-commissioned rapid review of 89 studies examining initiatives to protect children on the move from the across the world, we have identified six persistent challenges and ways to strengthen evidence to better protect children on the move.
1. Lack of rigorous impact evidence
Of the studies reviewed, surprisingly few reflected rigorously on the impact of project activity. Less than half included a control group or incorporated some form of pre- and post-intervention comparison, though most at least gathered insights from different stakeholders. Relatively few studies presented data on the scale of change, and even fewer discussed whether changes were statistically significant. This is an important limitation in understanding ‘what works’. Over a third of studies in our review were performance evaluations (which typically consider impact alongside other effects), rather than impact evaluations (where the sole focus is impact), which may explain this gap.
Potential response: By planning more rigorous evaluation of impact at the project design stage, practitioners can build a collection of relevant data for both impact and performance evaluations. Where collecting such data is not possible, practitioners can at least strengthen qualitative reflection on impact of activities, and the extent to which they can be attributed to a particular initiative.
2. Evaluations of policy and system-strengthening activities rarely discuss impacts on families and children
These initiatives are often evaluated in terms of intermediate indicators such as staff knowledge, skills or attitudes, but not whether these changes actually lead to positive changes in the lives of children and families.
Potential response: Embed analysis of the impacts on children and families into routine project monitoring and reporting. In addition, light-touch case studies could probe how far policy reform initiatives, such as efforts to harmonise child labour legislation and response systems to protect all children in a given jurisdiction, lead to change in children’s lives.
3. Evaluations are not always well equipped to capture the impacts of projects that adapt to changing contexts
Where projects adapt to complex circumstances, evaluations cannot always capture the effects of changes in project design or implementation. This is especially true for studies focused on measuring the impact of specific approaches (e.g. communication campaigns or non-formal education courses).
The time lag between research and publication also means that newly emerging approaches are often not recognised in reviews like ours. For example, we would have liked to include more insights on approaches to support children in transit (such as the migrant caravan through Central America and Mexico), but most of the literature we found was based on advocacy materials without reflections on effectiveness.
Potential response: Make greater use of routinely-collected project data that can quickly identify any unanticipated changes to understand the impact of new practices. Plan evaluations with enough flexibility to explore the effectiveness of innovations that emerge. Performance evaluations are particularly well-placed to do so, given their often holistic perspective on multiple elements of an intervention. Reviews of good practice could also make greater use of case studies and key informant interviews to establish promising innovations.
4. Evaluations are sometimes unclear about project impacts on different groups of children
Evaluations often distinguished impacts by gender or age, but rarely both at the same time, or by other factors. Very few mentioned children with disabilities (an exception is UNICEF’s family centres in Gaza), and none mentioned LGBTQI adolescents and young people. Likewise, few explicitly specified working with ethnic or religious minorities (one exception was the Families and Schools Together (FAST) programme with Hmong political refugees in the USA). Thus, intersectional inequalities that affect children on the move and how projects are addressing them are often unexplored.
Potential response: Explore and record more thoroughly how different inequalities and identities affect child protection outcomes and any actions necessary to redress them. The majority of studies focused on adolescents, so it would also be valuable to focus on the effectiveness of strategies to protect younger children.
5. We found clear thematic gaps in the evaluation literature
Only three studies focused on approaches to protect returnees and none focused on children in transit (rather than arrival at a destination). Just two studies focused on initiatives challenging xenophobia, despite it being an important factor contributing to social cohesion and the risk of exploitation and abuse among refugee and migrant children.
Potential response: Support more thematic evaluations and low-cost learning studies, such as case studies on these themes.
6. Few studies took a participatory approach with children
Only nine of the studies we examined used participatory methods. Studies that are not participatory may have limited ability to capture unexpected insights and concerns about what would make the most difference to children on the move. They also miss opportunities to help build the skills of young refugees and migrants to carry out research.
Potential response: Make greater use of participatory approaches and involve children and young people in research teams (much guidance exists on how to best do this). This can both ensure young migrant and refugee perspectives are better reflected and can enable them to develop valuable skills and act as direct advocates for improved policies and practices.
Final reflections
Child protection systems are often under-resourced and internationally displaced children are not always eligible for support from national protection systems in host countries. Small-scale civil society initiatives – though innovative and often effective – are also insufficient compared with the level of support needed.
In the context of the huge humanitarian challenges displaced people face, strengthening impact evidence can seem like a luxury. We argue the opposite. It is a necessity to make sure scarce resources and evidence are used as effectively as possible. That might mean using light-touch approaches that maximise use of existing systems and data to strengthen evidence on the impact of initiatives. It may also mean investing in rigorous evaluation where there is a clear strategic case to do so. We hope that some of the approaches we have suggested here can contribute to this effort.
