News

The right medicine? – The evaluation evaluated

Development Policy04 Feb 2008Ellen Lammers

In the second part of this special report we launch the public discussion on the evaluation of the Dutch policy for Africa over the last decade. Here we bring together a selection of the views of 20 Africa specialists and development experts who commented on the comprehensive summary of the evaluation, its findings and recommendations. To read the full texts of the individual responses, click on the names on the right.

Development strategies for Africa have reached an impasse, while the continent is rapidly changing. That is the background against which the Policy and Operations Evaluation Department (IOB) of the Ministry of Foreign Affairs conducted its evaluation. Leo de Haan, director of the Africa Studies Centre in Leiden, commends the IOB for the perfect timing of its evaluation of the Dutch policy for Africa. Some describe the report as déjà vu, containing few surprises. Others believe it will stir up debate and bring into public view what have until now been merely rumblings backstage.

While the various contributors focus on specific aspects of the Dutch Africa policy, several point to wider issues. One is the fact that despite statements to the contrary, decisions continue to be made in The Hague (and Brussels and New York) rather than in Bamako, Kampala or Maputo. Another is that the Dutch government could sometimes be more courageous in refusing to follow international trends. There is no doubt, as David Sogge puts it, that the IOB evaluation ‘relieves us of some comforting illusions’.

But there are also some words of praise for the Dutch Africa policy. The IOB’s remarkable conclusions on debt cancellation had not yet been released when the contributors submitted their responses. But they did have some critical words about the IOB’s evaluation methods.

The IOB’s methodology

  • Nadia Molenaers: ‘When reading the IOB summary I started to doubt whether the evaluators looked at implementation or only at the disbursement of funds. Full disbursement is not necessarily a good proxy for performance. It might say more about spending pressure – or about being too soft a donor’.
  • Paul Hoebink: ‘The analytical framework used in this evaluation is disappointing. The background chapter on political-economic developments in Africa gets bogged down in a long listing of facts and themes. It would have benefited greatly from consultation of international (academic) literature. The chapter on Dutch policy intentions gives us all the policy papers, but without proper analysis of the real changes in policy goals. Because the goals are not sufficiently analyzed, the evaluation gets shipwrecked’.
  • Solveig Buhl: ‘The last OECD-DAC peer review of the Netherlands (2006) considered the independence of the IOB, which is responsible for evaluating broad policy and cross-cutting themes. That is a long-term strength within the Dutch system’.
  • Joan Boer: ‘The IOB evaluation lacks context and historical perspective. Have we learned from past mistakes? Do we perhaps do as much as is reasonably possible, given the circumstances of both donors and recipient societies? Evaluations like this should give insight into long-term trends as well into what the critical factors of success are. Only then will they truly add to the debate’.
  • Simon Maxwell: ‘The Dutch are good at being reflective about what works and what does not. There has been nothing comparable in the UK in terms of evaluation’.

Agriculture – the new aid orphan?

In the mid-1990s Dutch support to rural development was largely dropped. This IOB finding touches a raw nerve with most contributors. To Anders Danielson (Ministry of Foreign Affairs, Sweden), it is one of the report’s most important findings. He points out the irony that investments in rural development (particularly infrastructure and agricultural extension) are known to yield the highest returns of all measurable aid activities. George Ayittey, an economist at the American University in Washington, calls it ‘ludicrous’ that ‘neither the Dutch government nor the recipient African country knows where the poor are’. The overarching goal of Dutch aid is poverty alleviation. The vast majority of Africans live in rural areas. ‘Come and have a look’, he seems to say, that is where the core of the poverty problem is! A particularly sore point is that Dutch support to rural development virtually disappeared without this being a well argued or even conscious policy decision.

The Dutch are not the only donors who have neglected agriculture, say Solveig Buhl (OECD), Oliver Morrissey (University of Nottingham, UK) and Han van Dijk (Wageningen University). The Dutch have simply followed international ‘fashion’. The consequences are grave. Ayittey notes that ‘today, Africa spends around $20 billion a year to import food – nearly equal to the amount Africa receives in foreign aid from all sources’. Han van Dijk relates the neglect of rural development to the fact that many marginal regions have become ‘the stage of rampant insecurity, protracted warfare and a stunning lack of progress in improving the basic indicators of human welfare’. He calls for rural development programmes that will help stabilize former conflict areas and reconstruct local governance capacity.

Buhl refers to a recent study by the OECD-DAC Network on Poverty Reduction, which confirms that agricultural growth, because of its influence on the rest of the economy, is the flame that needs to be kindled to reach economic transformation. There is hope that the trend of neglect may be reversed. The World Bank’s World Development Report 2008 is entitled Agriculture for Development. Several contributors, with more than a hint of irony, note that the World Bank’s words, spoken loud and clear, are usually obediently followed.

Joseph Semboja (REPOA, Tanzania) doubts that current social sector policies support agricultural development. ‘Today’s agriculture is highly knowledge, skills and innovation driven. In other words, it is driven by human resources. I believe our education systems should be much more geared to this understanding’.
George Ayittey (American University): ‘Much of the development resources and foreign aid were channelled into the modern sector in the frenzy to “industrialize” Africa. In Côte d’Ivoire, for example, over 80% of development is concentrated around Abidjan. … This is the primary reason why Africa can’t feed itself’.

The favouring of social sectors

There is little doubt as to why productive sectors like agriculture were so low on the Dutch list of priorities. Dutch aid money was first and foremost – and generously – injected into social sectors like health and, especially, education. This again is an international trend. Buhl quotes OECD-DAC statistics showing that only 14% of all gross bilateral development assistance was spent on economic infrastructure and production in 2003–2004. The IOB evaluators find that the Dutch funds contributed to better social service delivery in some African countries. While no one would wish to deny praise where praise is due, some questions need to be asked.

Francis Nyamnjoh (CODESRIA, Senegal) wonders why it is that the Dutch government spent as much as €565 million on education in Africa over eight years, yet has little to show for it beyond increased access to primary education. Most contributors share the concern that while linking aid efforts to the Millennium Development Goals sells well to public opinion – clinics built and children attending school can be counted – this has led to a situation in which donors are falling over each other to fund social sectors. Some of them, moreover, are constantly on the lookout for the perfect spot to plant their national flag. The need to score successes on the MDGs has been an impetus for donor harmonization. But, says Joan Boer (Netherlands Ambassador to the OECD), the ‘this is my monument!’ attitude is more prevalent than is readily admitted.

Nils Boesen: ‘The point is not whether what the Netherlands did was partly good, partly less good – the point is that the donor community tends to move in the same direction until they discover that this was not exactly working as expected, whereafter they run collectively into a new direction. Herd behaviour, frankly’.
Joseph Semboja: ‘A perusal of poverty and development strategies of many, if not all, African governments will show that there are no explicit strategies for growth. Many have focused on development in the social sectors, which is considered independent of economic growth. This kind of separation makes social development unsustainable in the absence of donor funds. It also ignores income poverty as an important item on the policy agenda’.

General budget support

As well as funding specific sectors, the Dutch government has also provided general budget support to selected African countries. The amount of GBS increased rapidly in the period under study, to become the third largest spending category. In theory, the money goes to countries that uphold – at the very least – some principles of good governance. But the IOB criticizes the Dutch government by concluding that GBS has become a ‘political instrument’. The minister often decided who the deserving recipients were, regardless of the outcome of the track record, if her private wisdom told her so. Several contributors pick up on the IOB finding that this track record is controversial in itself. Morrissey, a professor of development economics, asks how Dutch aid can ever hope to be effective if its criteria for giving GBS are not made available to other donors or the recipient countries.

Civil society sidelined?

The aid policy of the Dutch government has increasingly excluded civil society. Political scientist Brian Pratt (INTRAC, Oxford) believes that this is due to the ‘excessive focus’ on government-to-government aid, coupled with a failure to really deal with governance issues. Joan Boer agrees. ‘Responsibility should lie with recipient societies, not with their governments only. It may be understandable that the Dutch ministry sees the governments in Africa as its primary counterparts, but this is not how it should be. This is a painful finding for the Dutch, who have always stressed that civil society is crucial for the advancement of democracy in Africa’.

Nadia Molenaers uses the IOB criticism to question the role of Dutch civil society, and parliament, in the wheelings and dealings of Dutch aid. Based on her experience in Belgium, she doubts that they are well enough informed, let alone involved, in the political matters tied up with the so-called new aid architecture. Steven Ndegwa and Doris Voorbraak, both governance specialists at the World Bank, single out the observation that nearly a third of relevant policy documents were not shared with parliament. They frown on this ‘fundamental failure of democratic accountability’ in the Netherlands, since it inevitably affects civil society in Africa. After all, citizens in developing countries can best hope to affect Dutch policy and practice through intermediaries speaking to political rather than bureaucratic power.

To show that many countries find themselves in the same boat, George Ayittey gives the example of a bill proposed by a Canadian parliamentarian to improve its government’s accountability. One of the bill’s provisions, which would have enabled ordinary people in recipient countries to petition the Canadian minister about the administration of Canadian aid in their countries, did not make it through the negotiations.

Nadia Molenaers (University of Antwerp, Belgium) challenges the IOB suggestion that a yes or no to budget support should be based solely on the outcome of a technocratic instrument. Budget support is political, because effectively it is about giving money to a political party in power. Danielson refers to the example of Tanzania, a country he knows well. ‘Tanzania went from being regarded as a bad guy with rampant corruption and virtually non-existent control mechanisms for public funds, to an aid darling to which large amounts of GBS were disbursed in just a couple of years. I wonder what, exactly, had happened in Tanzania’s public finance management system over these five years that motivated such a change in attitude among donors?’

Morrissey has a point of criticism that concerns not the Dutch implementation of GBS, but the IOB evaluators. He suggests that they did not quite grasp the idea behind budget support. ‘The IOB appears to treat GBS as a poverty reduction instrument, which it was never intended to be. GBS is nothing more than a mechanism for delivering more effective donor support’. Indeed, there is no guarantee that the way African governments spend the funds will have a positive impact. Molenaers reminds us that their commitment to poverty alleviation cannot be objectively measured. Donors like flexible funds that are easy to spend, but the flipside is that as soon as the funds are disbursed, the donor loses track. Will the Dutch approach to budget support fall into the same trap as did the approach to policy reform in the 1990s – which is now heavily criticized by its former champion, the World Bank? Wil Hout (Institute of Social Studies) warns that ‘policies are meaningless unless they are backed by controls that make the policy actors accountable’.

Ernest Aryeetey (ISSER, Accra), finally, wonders whether the evaluators are too pessimistic about GBS: structural improvements in terms of poverty alleviation take time to happen.

Bad governments, fragile states

The IOB evaluated the Dutch approach to conflict countries. No gratuitous exercise given that growing political instability and deteriorating security situations are ranked high on the shortlist of the most pressing problems in Africa. The issue here is whether Dutch aid should support fragile states, and if so, how? One problem is that the governments of fragile states are not trusted. Aid money ends up running their patronage system. With a high risk of Dutch taxpayers’ money leaking away to all sorts of inglorious projects of African political elites, support to such states is hard to justify to the public, writes Han van Dijk. Yet Bert Koenders, the current Dutch minister for development cooperation – whose policies are yet to be evaluated – has made ‘fragile states’ one of his spearheads. Van Dijk, de Haan and other contributors commend his political courage, but they warn against the inevitable pitfalls.

One of these pitfalls, says Leo de Haan, is that even in fragile states donors cannot escape ‘engaging in political coalitions’, or, in other words, they risk getting their hands dirty. And yet van Dijk stresses that there is ‘no sensible argument’ for excluding the most problematic countries from Dutch aid. Even less so if one realizes that, as he writes: ‘The focus on so-called well governed countries is arbitrary, when the actions of those countries outside their national territory are not taken into account. The havoc created by Rwanda and Uganda in eastern DR Congo should have been stopped much earlier. Instead, both countries were rewarded for their economic growth and adherence to international macro-economic conditionalities’.

It is tacitly accepted that politics cannot be excluded from development cooperation, even if this is not always communicated frankly. In this context, Brian Pratt wonders where the huge sums provided by the Dutch government for humanitarian aid actually went. Referring to the Dutch Sudan policy, which took shape in a rather ad hoc way during Minister van Ardenne’s many visits to Sudan, he asks whether this perhaps amounted to the political use of the aid programme for other ends.

The recent interest in fragile states has its roots in politics. As an illustration, Stephen Ellis (African Studies Centre, Leiden) notes that in the 2002 US National Security Strategy, President Bush stated that America ‘is now more threatened by weak and failing states than … by conquering ones’. And here lies another potential pitfall. The IOB concludes that, despite the teething troubles and the time it is taking, the Dutch government’s ‘3D approach’ to integrating diplomacy, defence and development cooperation, is making headway. But David Sogge is unconvinced. He recalls that in 2005 an inter-ministerial commission concluded that Dutch foreign policy was anything but ‘joined up’. ‘With at least 13 different official departments, each pursuing its own programmes with great verve, managing foreign policy is like herding cats’. According to Stephen Ellis, efforts to reach agreement on a common strategy across the ministries will always be thwarted as long as these ministries do not agree on why fragile states deserve our attention. ‘For development activists, the hallmark of fragile states is that they do little or nothing to help alleviate poverty. But for military and security officials, their key characteristic is the possibility that hostile armed groups could use their territory for military purposes’.

Francis Nyamnjoh believes that the focus on certain forms of poverty has been to the detriment of others. ‘Too little attention is paid to what I would call, for lack of a better term, cultural poverty. People’s perceptions are shaped and reshaped over time. Therefore, policies are needed that aim at eradicating cultural poverty in Africa, both within and outside of the geographical area called Africa. Africans are crying out for the space and means to tell the story of African creativity in dignity’.

Donor-driven aid after all?

‘Too much policy, too few priorities’ is the IOB verdict. Danielson writes that the inability to formulate a limited number of priorities and stick to them – or, ‘the desire to be everywhere, always’ – is a weakness that other small and generous donors like Sweden and Norway suffer from too. Wil Hout (ISS) refers to another IOB evaluation which suggests that a ‘disturbingly large’ set of criteria for country selection enabled ministers to pick countries and always have at least some criteria at hand to support their choices. Besides, due to the lack of clear priorities, parliament cannot check whether the decisions are based on previously agreed policy principles.

There are more downsides. Ndegwa and Voorbraak find the Dutch failure to match practice to policy disturbing because consistency and reliability are the very things we want from developing countries. ‘Practice what you preach’ should be the minimum rule of thumb, and aid effectiveness an ‘ethical imperative’. In line with this, Maxwell proposes a theme for international lobbying: the need for better structures to underpin mutual accountability. Rich countries must be accountable to poor ones, just as the poor are to the rich.

Stephen Ellis: ‘A point often overlooked is the degree to which the development policy of a rich country like the Netherlands has, in the end, to satisfy a domestic constituency. This is the main reason for some of the twists and turns of Dutch policy, such as the decline in support for rural development in favour of other policy areas that appeal more to the development lobby in the Netherlands itself or even to the broader public’.

Brian Pratt writes that due to frequent new initiatives, nothing is ever given a chance to show fruit. Joan Boer is also highly critical of what he calls the ‘and now for something completely different!’ approaches of new governments. ‘Africa is not a garden centre where pots can be arranged and rearranged to please the donor customers’. But in practice it is not always that bad, says de Haan. Each new minister may wish to change tack, but below decks the bureaucrats are steering a steady course. Nevertheless, de Haan also acknowledges that each new minister wants to leave her or his mark. ‘Politics in the Netherlands matter more than Africa matters’.

Another issue that the contributors jump at – even if no one is surprised – is the IOB’s conclusion that the Dutch government ‘paid little more than lip service to ownership’. The empty shell of ownership is an issue in most international development assistance, say Hout, Sogge and Pratt. Danielson relates this to the current discussions about the loans that China is giving Africa. Loans that often simply replace old written-off debts. The ease and eagerness with which the contracts are signed say something about the myopia of certain African leaders, Danielson writes, but this also points to a conflict that the evaluation does not bring out. ‘Stricter conditions on how, when and where recipients take up new loans after debt relief should be weighed against the issue of ownership and the recipients’ right to their own development vision’.

The IOB recommends ‘a realistic approach to ownership, without rhetoric’. David Sogge is skeptical. Hard, operational definitions of the concept of ownership have not yet seen the light of day. Development partners (‘another prettifying term’ according to Sogge) do not operate the real policy levers. Donors, including the Dutch, have remained in the driver’s seat.

Footnotes

Unfortunately, due to the age of this contribution and several migrations to online content management systems, the footnotes in the text may have been lost. The footnotes below are listed in its original order of appearance in text.
  1. tOECD-DAC (2006) Promoting Pro-poor Growth: Agriculture. Paris: OECD
  2. World Bank (2007) World Development Report 2008: Agriculture for Development
  3. World Bank (2005) Economic Growth in the 1990s: Learning from a Decade of Reform. Washington, DC, p. 258
  4. Ellen Lammers (2007) How politics ticks: Debating politics and povertyThe Broker, 4: 13–15
  5. Quoted in Stewart Patrick and Caysie Brown (2007) Greater than the Sum of its Parts? Assessing ‘Whole of Government’ Approaches to Fragile States. New York: International Peace Academy, p.33
  6. Wil Hout and Dirk-Jan Koch (2006) Selectiviteit in het Nederlandse hulpbeleid, 1998–2004 ( Selectivity in Dutch Aid Policy), IOB Working Paper. The Hague: IOB
  7. 7See, for instance, Charles Gore (2004) MDGs and PRSPs: Are poor countries enmeshed in a global-local double bind? Global Social Policy, 4(3): 277-83