Speaker:
Alison Evans, ODI Associate
Chair:
David Booth

Alison Evans's presentation opened with a series of observations from the evaluation of The World Bank's Poverty Reduction Strategy in the 1990s which she headed for OED in 1998/00. These were:

• The Bank's strategy, based on the WDR 1990 (broad-based growth, social sector investments and safety nets), was relevant to the task of poverty reduction, especially given the state of knowledge about poverty in 1990.
  
• In many ways the Bank has been successful in implementing the strategy - note the availability of poverty data in many countries, the usefulness of poverty assessments in stressing the multi-dimensionality of poverty and the extent of investment in health and education sectors - but there are also important areas of weakness. These include: difficulty in translating the concept of broad-based growth into practical country-led policies and programs; the relative neglect of the private sector and of governance; inadequate attention to the important synergies between the three-legs of the strategy; and inadequate attention to monitoring and evaluation.
  
• Since 1990 a lot has changed in the Bank and by the time the evaluation was up and running, three major initiatives had begun to change the way the Bank does business - the Strategic Compact, the CDF and the PRSPs. This made evaluation difficult (the goal posts were constantly moving). Nevertheless they confirm, at least one key observation, that the role for aid is always going to be modest unless there is active partnership with poor countries and between donors and unless countries are given more room to lead the fight against poverty.
  
• The WDR 2000/1 offers some new insights into what can be done to reduce poverty more effectively. These were important additions to the 1990 strategy, but were also part of a continuum of thinking that had developed over the decade. However, a key question still remained. How will we evaluate this new strategy in 2010? Were are the benchmarks, the evaluative criteria, and what is being done to ensure that M&E data will be forthcoming in the next decade to enable us to say something more concrete about the contribution of aid to country poverty reduction strategies.
  
• PRSPs may be the key here, a key question is whether aid can really create enough space to allow PRSPs to work effectively.

A number of points were raised in the discussion that followed. Firstly, it was asked whether the World Bank is capable of learning lessons, and co-operating with NGOs to use their expertise. Or does the Bank have a somewhat arrogant attitude in this respect? Secondly, the question whether aid can reduce poverty raised the issue of methodology. Is the problem lack of evidence, or are we asking the kind of question that is entirely impossible to answer in the social sciences? Thirdly, why has broad-based growth been so under-identified and so little understood? Given the fact that the Bank has both research and operational departments, achieving some clearer understanding of the concept should have been a feasible task. And why are we not able to tell whether aid can reduce poverty? Even though aid is only one component in strategies aimed at reducing poverty, ways of investigating these issues do exist.

Alison Evans replied that the methodology for evaluating the effect of aid was indeed inadequate, at least below very high levels of cross-country aggregation. While there have been methodological improvements at the program and project level, adding these effects up was a major problem. At both levels, impact evaluations can be, and indeed are being, done. However, evaluations are costly, which is one reason why impact evaluations are not done more often. The stock of impact evaluations is very small compared to the entire WB (and aid) portfolio.

In terms of lessons learnt, the Bank is a complex organization. While one part may be focusing on lessons learnt, other parts are not. Relations between OED and the rest of the Bank are often conflictual. Some lessons are taken on board; however they do not always feed through to practice. The debate on broad-based growth is a case in point. Despite some very important research on growth and poverty at the top of the organization, many Bank staff working on specific sectors and projects are not aware of it, or find it hard to relate to their area of expertise. The result is considerable uncertainty about applying the principles of broad-based growth in everyday operations.

During the next round of questions, it was asked whether, aid being expensive to deliver, the focus should be on contributing ideas. It was furthermore claimed that maybe the question we should be asking is what kind of aid is found to be effective, taking advice from local communities and NGOs. The next question was whether the goalposts have moved. Has the perception of what is being measured, and thus what counts as poverty reduction, changed between 1990 and today? The focus of 2000 is very much on exclusion, not only exclusion of individuals, but indeed of whole subsections of populations. This implied different criteria of success.

ODI's director, Simon Maxwell, was concerned about the political implications of being sceptical about aid effectiveness. There is an election coming up in the UK, and we should be wary of giving weapons to those who disagree with giving aid in the first place. Instead, we should be making a positive case for the kinds of aid that are known to work better than others - for example offering "10 things to do" to improve aid.

Alison Evans agreed to the suggestion that ideas, not only money, do matter, and certainly money without ideas does not do any good. But as a lending institution, the Bank cannot live on ideas alone; money remains an important factor. There are many examples of things shown to work in a variety of settings, and more examples could probably be found. However, many evaluations by NGOs and others show that the impact of projects is more often than not localised and not especially far-reaching. With regard to the goal posts, Evans agreed that they have moved, and that there have been many conversations about what we are measuring. Although the limits of headcount measures are acknowledged, there is no consensus as to what we could effectively measure instead.

As to what kinds of aid work better than others, Evans was not particularly convinced that "10 things to do" is the most productive path; this could indeed be counter-productive, suggesting a mechanical, formulaic approach to, for example, the PRSP process. We can build a case for aid, but we do have to be humble in how we go about it.

The chair then suggested that a better question to ask is to which extent countries, with some support from aid donors, can be effective in reducing poverty. A few more success stories might be sufficient to make this a compelling political case for aid. It was suggested by another participant that the new WDR is distinguished by not offering a general strategy, instead transferring responsibility to movements and forces within countries; but this posed problems of local capacity. Another speaker thought that very few African governments are seriously committed to poverty reduction. Should strict, selective criteria be attached to aid, we would be unable to support many African countries. This would make it difficult to spend even existing levels of aid. This led to concern at the extent to which poverty reduction has become the overarching focus of aid, to the exclusion of other purposes. A final speaker was concerned that the aid debate should not distract attention from the potentially much greater effects of trade reform.

Evans did admit she had taken a rather narrow reading on aid, and that there are multiple aspects that were excluded from her discussion.