Policymakers face tough decisions when intervening in fragile and conflict-affected states. Donors are under pressure to spend resources wisely and effectively, but at the same time, there is a lack of evidence on which policies really work.

Conflict re-emerges across the world, which highlights the need to find long-term solutions to address instability. The relationship between political settlements (the political agreements upon which a state is based) and development is particularly vexing from an evidence-based perspective.

It is generally assumed that political settlements – and peace agreements in particular – affect the developmental outcomes of a state, since peace agreements are often concerned with state capacity. A more inclusive peace agreement may, for example, lead to a more balanced representation of the population in government. But how do peace agreements translate into actual development outcomes?    

Political settlements form a significant factor in the success of statebuilding processes, yet a 2012 evaluative study found that over 100 statebuilding engagements failed to apply such a lens to their analysis. A timely and necessary new discussion paper by the Institute for Security Studies (ISS) marks the start of a project that finds out how transformation can be measured in the context of political settlements.
It is a working paper, and readers are invited to make comments and suggestions on its methodology. The paper is part of the Political Settlements Research Programme, which is led by the University of Edinburgh and funded by the United Kingdom’s Department for International Development.

Because peace settlements are often seen as only one facet to longer, more complex peace processes, peace agreements can be seen as useful ‘markers’ for measuring progress on developmental outcomes. Since little quantitative data exists, the programme is currently establishing a database on peace agreements.

The data will be coded to allow for comparisons of elements of inclusiveness, development and rights between these peace agreements, and it will provide up-to-date, comprehensive and detailed quantitative information. The database will permit subsequent transformation to be measured according to different criteria, such as reductions in violence, improvements in infant mortality rates and education.

Measuring transformation is also tricky, however. Current debates often refer to transformation in the context of a so-called fragility-resilience spectrum, where state fragility refers to ‘a period of time when sustainable socio-economic development requires greater emphasis on complementary peacebuilding and statebuilding activities such as building inclusive political settlements, security, justice, jobs, good management of resources, and accountable and fair service delivery.’

Greater resilience, on the other hand, is an elusive goal that fragile states aspire to. It is defined as ‘the ability of social institutions to absorb and adapt to the internal and external shocks and setbacks they are likely to face.’Transformation is the change across this spectrum. 

Statebuilding does not cover all aspects of transformation, but peace agreements often deal with state function. The new Institute for Security Studies (ISS) paper suggests that measuring transformation along the core functions of the state is a necessary starting point.

Current debates – such as that of the New Deal for Engagement with Fragile States, which proposes peacebuilding and statebuilding goals; the Post-2015 Sustainable Development Goals (SDGs); and the global development agenda – often emphasise the need for additional, country-specific quantitative assessments of fragility. Yet many African countries are often unable to gather even basic demographic data on a regular basis. Information on issues such as ethnicity may also be politically sensitive and, if not used properly, could inflame tensions.

A number of databases currently provide information on different indicators of state capacity – in thematic areas such as security, justice, legitimate politics, economic foundations and revenues and services. These include the World Bank’s World Governance and Country Policy and Institutional Assessment data, the United Nations Development Programme Human Development Index, Polity IV, Freedom House data and Transparency International’s Global Corruption Barometer.

Some of these datasets draw on expert opinion, while others are based on surveys or even opinion data. Although perception-related data is important in assessing state function, this is limited at present.
These datasets also vary according to country coverage, time spans of assessment, aggregation of data and data gaps – making some more suitable than others. They do, however, provide important starting points for measuring transformation. There are also composite indices that measure variations of fragility and resilience.

The Institute for Economics and Peace’s ‘Pillars of Peace’, for example, uses a number of existing datasets to create different measures of a peaceful society. Composite indices are tempting to use: they give the impression that by including a greater number of indicators, the many nuances of transformation are better reflected. At the same time, they may not use the best choice of proxies since they require that the underlying datasets are appropriate and up to date.

Each indicator, and its underlying data, requires careful analysis. Even more important, considerable research and expertise is needed to ensure that the weighting and balance of each dataset presents a meaningful relationship, as opposed to being included based on preconceived notions of relevance.

The ISS working paper outlines a number of different datasets and indicators for measuring transformation. The aim of the paper is not to determine the specific indicators of transformation, but to rather to explore the possible range of indicators for further discussion at a later stage. Once determined, different types of peace agreements can be mapped according to different transformative outcomes, which can influence policymaking in the future.

Measuring transformation in the context of political settlements will not be easy. But if donors are to spend their much-scrutinised money more effectively, they need to consider where and how to get the best bang for their buck. This involves analysing how political settlements can influence human development trajectories. The working paper hopes to stimulate discussion on this topic and invites comments and suggestions.

• Amanda Lucey is a senior researcher at the Institute for Security Studies’ Conflict Management and Peacebuilding Division in Pretoria. This article first appeared in the ISS website.

Picture courtesy of Institute for Security Studies: www.issafrica.org