Apparently transparent – Does accountability deliver?

Transparency and accountability initiatives aim to combat corruption and inefficiency, and improve how aid is channelled. How effective are these initiatives, and how can their impact be measured? 

Transparency and accountability initiatives (TAIs) have taken democratization, governance, aid and development circles by storm. ^1 2 They have flooded into the void that existed between citizen-side efforts to promote participation and voice, and state-side efforts to promote development effectiveness and democratic governance.

Developmentalists hold that accountability will mend the leaky pipes of corrupt and inefficient service delivery, and channel aid more effectively. As a consequence, development initiatives will produce greater, more visible and sustainable results. Democracy scholars and proponents hold that democracy now needs to deliver on its promises – including better living standards – and new forms of democratic accountability can help in this.

Traditional inadequacies

Traditional political accountability suffers from inadequacies such as administrative bottlenecks, weak incentives and corruption. These inadequacies limit the effectiveness of state-led mechanisms, particularly for poor people. Myriad multi-stakeholder, citizen-led and ‘social accountability’ approaches have emerged in response. Citizens, communities and social organizations have bottom-up monitoring systems, citizen report cards, social audits, grassroots analysis and research, public hearings and complaints mechanisms. Led by social actors rather than traditional political ones, this social accountability is nonetheless deeply political in terms of stakes and impacts.;

Social and citizen-led accountability moved to centre stage when the World Development Report 2004 identified service delivery failures – in education, health, sanitation and energy provision – as accountability failures. For example, primary education facilities, often under-funded, were found to suffer additionally from teachers being poorly motivated and often absent or ineffective at their jobs. Increased public spending on primary education would achieve little in school systems mired in corruption and political patronage, where teachers’ salaries continued to be too low and paid late or not at all. The report advocated direct interaction between service users and providers to address these problems, instead of the ‘long route’ of elected representatives and public officials seeking accountability from providers on users’ behalf.

The rise of transparency and accountability

Broadly, TAIs seek to achieve one or more of the following impacts:

• Increase development effectiveness – the ‘developmental outcomes’ case
• Improve the quality of governance – the ‘democratic outcomes’ case
• Empower poor citizens – the ‘empowerment’ case

In the service delivery field, many accountability initiatives hinge on getting governments and service providers to publicize information or make their budgets transparent, sometimes through oversight by user-citizens. One example of such an initiative is the famous Indian people’s organization MKSS, which has inspired many modern social accountability initiatives, in India and abroad, via its mass campaigning, and its naming and shaming of corrupt officials.

The field has thus overlapped with simultaneous developments in the access-to-information field, which was already a burgeoning area of – often legal – advocacy linked to social mobilization. It merges too with budget accountability work, which has evolved prolifically since the mid-1990s.

TAIs have spread even further more recently. They have reached the extractives sector, where methods like accountability rankings and indices have been borrowed from the access-to-information and budget fields, and other purpose-built approaches have been developed.

The Extractive Industries Transparency Initiative, for instance, is a multi-stakeholder coalition of companies, investors, governments, and local and international civil society actors. Launched in 2002, it sets standards for disclosure and transparency on extraction and export revenues, and monitors compliance. ³ Publish What you Pay is a global civil society network that campaigns and advocates for disclosure of similar information, to ensure that revenues from oil, gas and mining activities benefit local populations, not only companies. ⁴

In development aid, there are long-standing concerns about the fundamental inequality of aid relations, which tends to make them opaque and unaccountable. International NGOs have sought to use partnership protocols and accountability standards to address these concerns. ⁵ Official aid Donors have emphasized ‘mutual accountability’ in their general commitment to enhancing aid effectiveness – although lately the emphasis lies heavily on recipients’ and intermediaries’ accountability to official funders.

The latest expression of accountability concerns in the aid sector is a wave of ICT-based aid transparency initiatives. Features of this ICT transparency wave are now seeping into the climate change field, as alarm grows about huge international public funds pouring into mitigation and adaptation efforts in developing countries without an adequate, purpose-built architecture in place. The ICT initiatives are borrowing heavily from the aid architecture, which we know to be deficient, not least in terms of its inherent inequalities, lack of accountability and opaqueness already mentioned.

Beyond the buzz

Yet as they become mainstream, accountability and transparency risk losing their meaning and becoming buzzwords. There are signs that their political edge is becoming blunted and their democracy-deepening potential neglected as they are applied in pursuit of narrowly conceived developmental outcomes. A stress on demonstrable outcomes in the form of efficient service delivery has diverted attention from how citizen voice and participation could have shaped policy design and priority setting in the first place. A focus on the tools threatens to eclipse the vital issues of context and relationships, leading to the mechanistic application of gadgets and kits to situations where they are inappropriate, or at the expense of examining power relations between actors.

After a decade of the spread of TAIs, donors, development practitioners and researchers are asking what all this accountability and transparency activity and expenditure are achieving.

Are TAIs proving effective in terms of achieving their stated goals, such as providing service users with relevant information, or shaming extractive industries into paying their taxes? Beyond these immediate effects, are they achieving further-reaching, ‘second-order’ impacts, such as material benefits in people’s lives due to better services or better aid? Are they helping to establish democracies, which serve their poor constituents better, or to empower citizens by making their voices heard?

What methods are useful for assessing this? How do early assumptions about connections between participation, transparency and accountability look in the light of a decade’s experience? Do citizen-led TAIs work effectively with state actors? Do they manage to change state institutions, and if so, which and how?

What we can say

What we can say about TAIs is that existing evidence shows that under some conditions, some TAIs create opportunities for citizens and states to interact constructively, contributing to greater state responsiveness to citizens’ needs, better budget utilization, improved service delivery and more empowered citizens (see box).

But there is a lot that we cannot say, on the basis of existing evidence, about the impact of TAIs.

Overall, the type and quality of evidence is uneven, piecemeal and scattered. Many studies focus on just a few initiatives. As yet there is a dearth of rigorous ‘meta’ or secondary reviews that look across a range of evidence to draw broader, more solid conclusions from specific cases. Many TAIs are just too new for any conclusions to be drawn about impact, let alone sustained impact. Most studies focus on the first level of impact – the effectiveness of the initiative itself – without showing links from the initiative to broader developmental, democratic or empowerment aims.

None of these observations constitute arguments against transparency and accountability (T&A) work. But they do show that more robust evidence is needed to make the case for T&A convincingly. This is vital for its survival now that evidence-based policy and results-based management have pervaded the development sector, and in the post-financial crisis context of shrinking aid spending.

How can we enhance the demonstrable impact of T&A work? Three major challenges come into focus for researchers and practitioners.

Aims, claims and theories of change

The first challenge arises from the fact that in TAIs, aims vary, working assumptions are obscure and untested, and theories of change are rarely adequately spelt out.

To discuss the impact of TAIs, or what they have achieved, we need to be clear about their aims. Did they seek development? Deeper democracy? Empowerment, or what they sought to achieve? The intended impact of a TAI pursuing developmental outcomes probably differs from one whose goal is to empower citizens. This obvious point is not always grasped as centrally as it should be: some TAIs are not explicit about their intended outcome, and some impact assessment approaches are too vague or general.

TAIs often leave unclear whether the immediate outcome was an end in itself, or seen as a means to an end. Some initiatives take it as a given that transparent aid data will lead seamlessly to more accountable and more effective aid, even though the evidence to date does not support this assumption. The most common buried, untested assumption by far is that transparent, accessible information will generate accountable policies, budgets and state behaviour.

Any TAI is just an intermediate step in pursuit of the desired impact. However, how exactly that impact is to be achieved – or the ‘theory of change’ – is often left unsaid. Most TAIs miss any realistic appraisal of how inputs might translate into the desired outcomes.

Take the case of an African NGO that wants government spending to be fairer to rural and socially marginalized populations. It pursues this aim by lobbying for budget proposals to be published on government websites. The causal pathway that could lead from the lobbying (the input) to the outcome (changed budget allocation and execution) is something like this:

• The budget proposal document is eventually published – that is, the transparency initiative has been effective, but it has not yet had its desired impact
• Concerned organizations, citizens, parliamentarians and journalists access it online
• They put pressure on the finance ministry’s executive and budget office to change current budget allocation patterns
• The budget office responds to the diverse pressures and changes budget allocations
• The resulting budget proposal is actually executed as set out in the proposal, with proportionately more spent on rural and marginalized populations

None of the steps in a causal chain is a foregone conclusion. All involve assumptions and risks. The NGO could mitigate some of these, for example, by forming alliances with organizations that focus on social mobilization in villages, or working directly or through coalitions to promote budget execution monitoring. The NGO might report to its INGO funder that the budget proposal is now available online. That might prove the effectiveness of the NGO’s lobbying, but not its impact: for that, a very different strategy would be required, almost certainly involving additional stakeholders.

Not the next logframe

All in all, few initiatives are set up so that they can provide concrete evidence of having advanced their ultimate aims.

There is a difficulty here. Theories of change should not be presented as ‘the next logframe’, as today’s must-have accessory that qualifies development initiatives as fundable and workable. The very language of theories of change is alien and off-putting to many of the people working in development. To realist practitioners who see their assumptions constantly calling for revision as non-linear, complex realities unfold around them, a theory of change – or of anything – may sound too fixed and restrictive.

But at a basic level, a failure to spell out assumptions of how one expects change to happen can inhibit an initiative’s effectiveness by limiting its focus. It can also make impact assessment elusive or impossible: against what would we assess impact and explain its attainment or non-attainment? And, as practitioners know, the collective articulation of a theory of change between all involved actors offers invaluable opportunities for negotiation and contestation around the proposed change effort. This helps to deliver a more relevant, feasible and sustainable process.

How do we know what we know?

Impact assessment in the field of T&A to date has added to a rich array of methodological approaches already deployed in other fields. These include both qualitative and quantitative approaches.

Assessing the impact of complex, multi-actor change processes is difficult in any field. The relatively young field of T&A work, which seeks to yield change in complex socio-political relationships, is no exception. Assessing TAIs’ impact means facing up to a range of methodological challenges.

To start with, the available evidence is limited in quantity and uneven in quality. Observations of correlation are all too often mistaken for causal connections. The effects of single factors are easier to pinpoint and trace than the interaction of several factors, yet this interaction is often the key that unlocks outcomes, as is demonstrated by the causal pathway example above. Formulating manageable indicators to capture concepts such as ‘empowerment’ or ‘denser democratic engagement’ is a struggle for even the most incisive thinkers and practitioners of impact assessment. And then there is the problem of attribution versus contribution, bemoaned by Irene Guijt ¹¹ as ‘a recurring headache for those engaged in multi-actor, multi-location and multi-strategy change work’.

At one level, to resolve these challenges we need more of the same. There are a number of good, specific studies that use a range of methods, but there are not enough of them, across enough settings or enough methods, to point unequivocally to overall patterns or to draw higher-order conclusions. At another level, the body of evidence needs not only to grow but also improve – better-designed studies conducted on better-designed TAIs.

The state of the evidence could be improved by developing impact assessment approaches pioneered in other fields that are sensitive to complexity and draw on combined methods. Untapped potential in user-centred and participatory approaches could be explored further. More baselines could be used, such as contextual ‘outset analysis’ rather than slavish logframe compliance. Comparative in-depth research across contexts and across TAIs can be conducted within multi-case studies. The capacities of researchers and practitioners can be strengthened for developing and building on innovative approaches.

Donors and practitioners of accountability and transparency work could ask these questions early on:

• Does the intervention or initiative articulate a clear causal pathway? Does it disentangle common assumptions about the links between transparency, accountability and participation?
• If adapting, replicating or scaling ‘successful’ applications of particular tools or approaches to other settings, does it take into account the reasons for their success in the original context? Has it considered issues of timing, sequencing and durability?
• Does the strategy take into account complex, contextual factors, including capacities and incentives on both citizen and state sides of the equation, and mechanisms that link the two?
• Does the design contemplate how impact will be assessed? Does it build in methods of analysis appropriate to the purpose of the impact assessment, given the audience, the level of complexity involved and positions of those assessing its impact?
• Does it include methods for tracking change over time, including reference to a descriptive baseline, methods for learning by comparison with other, comparable initiatives?

Beyond the state-citizen dichotomy

There is, then, little evidence to support generalizations of the kind ‘an initiative of type x produces outcome y’. A more useful question to ask appears to be, ‘Which factors – both enabling and disabling – shape the possibility that TAIs will achieve their stated impacts in a particular context?’

Despite the unevenness of the evidence base and its limitations, there are common factors that shape the impact of TAIs. These show that TAIs are not only mechanisms or instruments, but relationships, involving power dynamics and patterns of behaviour and attitudes across both the state and society sides of the governance equation.

On the ‘citizen voice’ or demand side, one key factor is the capabilities of citizens and their organizations to use the information that has been made transparent to push for greater accountability. The Indian RTI law would be much less effective if the process of securing it had not involved awareness-raising and the mobilization of vast masses of citizens. This made it possible to put it to the test by submitting RTI applications and critically analyzing the results.

Also key is the extent to which TAIs are linked to broader forms of collective action and mobilization. The World Bank Inspection Panel, an internal accountability mechanism for preventing and redressing harm against people affected by Bank-funded projects, owes its successes partly to environmental and human rights NGOs, social movements and campaigners outside the Bank. They provide crucial demand, support and public visibility for the Inspection Panels’ interventions.

A final key citizen-side factor is the degree to which accountability, transparency and participation are embedded throughout all stages of the policy cycle, from how decisions get made to whether and how they are implemented – although these links are not yet well-enough understood.

On the ‘state’ or supply side, important determinants are the level of democratization or amount of space for accountability demands to be made. Others are the degree of political will from inside the state to engage with TAIs and the broader political economy. This includes legal frameworks, incentives and sanctions which affect public officials’ behaviour. The success of participatory budgeting in Brazil more widely owes much to the post-dictatorship governance context, which offers abundant, legally enshrined opportunities for citizens to engage with the state.

The most interesting current work delves into the interaction of the citizen and state sides. It explores how norms and cultures of accountability get changed on all sides, through cross-cutting coalitions of actors.

Jonathan Fox, in his 2007 book Accountability Politics, argues that ‘constructing accountability involves challenging the state, but also transforms the state’. ¹² He illustrates this by showing how successive state anti-poverty programmes aimed at Mexico’s rural poor allowed the socially and politically marginalized to develop autonomous collective action. In some regions, this succeeded in shifting the power balance between state and society. Essential elements proved to be the mobilization of masses and the building of coalitions between the social actors and allies within the state.

These observations about the importance of interfaces between states and citizens, as well as between different kinds of citizens’ organizations and social actors, is consistent with much current thinking in governance. ¹³ This thinking urges paying more attention to ‘accountability coalitions’ and networked approaches, changing norms and cultures of accountability in the state, private sector and civil society. It also advocates establishing links between the local, national, regional and transnational. It stresses the need to bring politics back in, by unpacking power and ‘political will’ and exploring accountability in relation to political parties, elections and regimes.

Conceptual dichotomies, useful for understanding what happens on one side or the other, lose some relevance when the interfaces between the two sides come into focus as the sites of vital change. After all, increasing accountability is about changing the balance of power between states and citizens.

More demonstrable impact

The evidence base is weak, but that does not mean that TAIs are not significant. The accountability and transparency community needs to work to enhance both the evidence available and the impact it has.

The synergies between transparency, accountability and participation could be understood and exploited better. We need to consider more carefully whether TAIs ‘travel well’ across context, method and issue.

Cutting-edge governance thinking, especially on networked governance and the interaction of the various levels from local to international, needs to be ploughed into the accountability and transparency field. And we must move beyond working on both sides of the governance equation in isolation, to building, strengthening and thickening the interfaces between state accountability agents and citizen accountability seekers.

Tammie O’Neil, Marta Foresti and Alan Hudson, authors of the 2007 report Evaluation of Citizens’ Voice and Accountability, conclude that uncertainty about the impact of accountability initiatives is but ‘a sub-set of uncertainty about the relationship between democracy and development’. ¹⁴ Notwithstanding the humbling uncertainties of social and political change, we now have some ‘known knowns’ and some ‘known unknowns’ for the accountability field. Concerted investment in knowledge and impact assessment in the accountability and transparency field is crucial if its promise is to be delivered.

The author would like to thank John Gaventa, fellow of the Participation, Power and Social Change Team, for his helpful comments on this article. This article is based on a review by the author and John Gaventa of the state of knowledge about the impact and effectiveness of TAIs. The review, and related documents, are available at http://www.transparency-initiative.org/publications.

References

Carlitz, R. (2010) Review of Impact and Effectiveness of Transparency and Accountability Initiatives – Annex 2: Budget Processes. Available at http://www.transparency-initiative.org/

David, R., Mancini, A. and Guijt, I. (2006) Bringing systems into line with values: The practice of the accountability, learning and planning system (ALPS), in Eyben, R. (ed) Relationships for Aid. Earthscan.

Department for International Development (2010) The Politics of Poverty: Elites, Citizens and States. Findings from Ten Years of DFID-funded Research on Governance and Fragile States 2001 – 2010. London: Department for International Development. Available at http://issuu.com

EITI (2010) Impact of EITI in Africa: Stories from the Ground. Oslo: EITI International Secretariat. View PDF.

Fox, J. (2007) Accountability Politics: Power and Voice in Rural Mexico. Oxford University Press

Goldfrank, B. (2006) Lessons from Latin American Experience in Participatory Budgeting. Paper Presented at the Latin American Studies Association Meeting, San Juan, Puerto Rico, March. View PDF.

Guijt, I. (2007) Assessing and Learning for Social Change: A Discussion Paper. Learning by Design/Institute of Development Studies.

Jacobs, A. and Wilford, R. (2010) Listen first: A pilot system for managing downward accountability in NGOs. Development in Practice 20(7): 797-811.

Jenkins, R. (2007) India’s unlikely democracy: Civil society versus corruption. Journal of Democracy 18(2): 55-69.

O’Neil, T., Foresti, M. and Hudson, A. (2007) Evaluation of Citizens’ Voice and Accountability: Review of the Literature and Donor Approaches Report. London: Overseas Development Institute

Reinikka, R. and Svensson, J. (2005) Fighting corruption to improve schooling: Evidence from a newspaper campaign in Uganda. Journal of the European Economic Association 3 (2-3): 259-67.

World Bank (2004) World Development Report 2004: Making Services Work for Poor People. Washington, DC: World Bank.

Footnotes