Expect the unexpected: focus on the unintended effects of international cooperation

It is striking how little attention is paid to the unintended effects of development cooperation in evaluations thereof, despite the OECD guidelines that stipulate that evaluators should look at these. A recent meta-evaluation of the evaluation quality of the largest bilateral aid agency in the world, USAID, showed that only 15% of it evaluations paid attention to the unintended effects of development cooperation. In the case of the Norwegian Agency for Development Cooperation, NORAD, only 40% of the terms of reference for evaluations requested evaluators to take unintended effects into consideration, which was often done in a superficial way. More and more it is acknowledged that this dearth of (evaluation) research on the unintended effects of international cooperation needs urgent attention. Dirk-Jan Koch, recently appointed Special Professor of International Trade and Development Cooperation at Radboud University, Nijmegen, states that two types of unintended effects – the ignored and the invisible – should be studied much more.

Firstly, let’s look at the ignored unintended effects. These are not ignored on purpose, but simply overlooked because of financial and methodological constraints or biases. Lau Schulpen and the author engaged in a literature review of 64 relevant studies to discover which unintended effects were overlooked. Only a very small portion of the literature analysed the unintended effects of international cooperation activities by so-called ‘intermediaries’ of international cooperation (e.g. local aid and peace workers). Of the 64 studies, only two deal with these. These effects fall outside the log frames of the donors and, hence, are often ignored by evaluators, who tend to have tunnel vision and are short of time. The preliminary findings of the research indicate that the effects are substantial, as over half a million jobs were found to depend on such intermediaries within international agencies and NGOs. Taking into consideration their dependents, the livelihoods of millions of people receive an unintended and unnoticed positive boost because of international aid: unintended effects can thus also be positive!

Additionally, wages between workers in the foreign-funded aid sector and the public sector differ hugely. In the Democratic Republic of the Congo (DRC), if a manager from the public sector moved from a job in the public sector to an American NGO, she would earn 10 times as much. If she worked for a bilateral donor agency, she could earn 27 times as much, or for the United Nations, 75 times as much. Hence, we need ask: What have been the effects on local governments who see their best employees move to other organizations? Insights into these ignored unintended effects would allow policy makers to capitalize on the unintended positive effects and reduce the negative effects.

Secondly, there are also invisible unintended effects. These are overlooked because existing rules and regulations prevent proper research into these effects, making them largely invisible. A lack of transparency hides these effects from sight. Some of the large infrastructure deals that Atradius, an export credit agency of the Dutch government, promotes, for example, such as ports and roads, might have potential unintended effects for local populations. However, until recently, relatively little information on these projects was made publicly available. A consortium of a Brazilian and Dutch NGOs filed a complaint two years ago with the National Contact Point (NCP) about the negative unintended effects of an Atradius dredging project in Brazil. The NCP deals with alleged human rights violations by Dutch firms overseas. The NCP indeed argued that too little information was made available, for instance on Atradius’ complaints procedure, and stated that more transparency was needed. Interestingly, the Dutch government argued that the NCP did not have the right to propose this recommendation, as Atradius was a state company and not a multinational enterprise. However, the NCP went ahead and published their recommendation, and now Atradius is preparing to roll out a new public disclosure policy. This shows the importance of open governments, especially when it comes to making invisible unintended effects more visible.

In sum, more attention needs to be paid to the unintended effects of international cooperation, be they positive or negative. To some evaluators this might be bewildering: where to start? An interesting methodology proposed by Jabeen is the unintended outcome evaluation approach (UOEA). This approach sets out three steps for evaluators: a) outline programme intentions, b) forecast likely unintended effects, and c) map the anticipated, and understand the unanticipated, unintended outcomes. Although this is easier said than done, nobody can be blamed for trying!