Summary

The third of six S&J seminars welcomed Todd Foglesong from the University of Toronto’s Munk School of Global Affairs to discuss ‘The Conscience of Indicators in Security and Justice’. The title was, as expected, only a fraction of what was discussed, setting the course for an introspective talk about why and how we measure what we do, how data is used, and for what purpose. It was primarily a discussion about politics or, more accurately, the process of converting statistics into “indicators” that advance political ends. The speaker invoked Cameron and Sarkozy, early Soviet planners, and numerous modern police forces to describe how indicators are an ‘extension of politics by other means,’ influencing how we allocate funds, assign responsibility, and mobilize hope. Clausewitz might have approved.

Indicators can be viewed as an assertion of power because in the fields of S&J they tend to be both centrally created and take agency away from people on the front line. Centralised indicators can have the effect of “instrumentalizing” people and ideas. This is why they are sometimes denounced as “micro-management,” the form that fear of other people’s discretion takes today.  We use indicators of “performance,” Todd claimed, in order to acquire control over “other peoples’ lives and political processes.”  We employ measurement to do our “moral dirty work.”

In fact the speaker suggested that there is a ‘masquerade of measurement’ in the S&J field today, with indicators covering up three distinct and deceptive purposes:

1. To attract support and investment

2. To dispel doubts others have about the value of our work

3. To ‘bolster our own bravado’ in the face of uncertainty

This is far removed from the task most indicators are expected to perform, that is help to solve problems by identifying and tracking real changes. Indeed this search for the truth appears facile when instead evidence from actual practices suggests that indicators are, perhaps inevitably, framed and interpreted subjectively to meet specific political or personal needs. Strong examples of this abound. Take the confirmation bias in the Victorian physician’s determination to prove the existence of a soul, for example. Observing the slight weight loss at the moment of death, the doctor inferred that this was a sign of the soul departing the body: quite a stretch of the data. People, it seems, measure what they can, and what they want to in order to find what they already know they are looking for.

Furthermore, security and justice indicators tend to be single issue measures of far more complex problems. In many ways indicators adhere to and encourage ways of thinking akin to a general or a doctor in how they attempt to identify, isolate and manage an issue. They’re also based on the assumption that any reductions in experiences of injustice engender a corresponding increase in justice. So, for example, an indicator that shows a decrease in court backlogs is taken to equate to improved justice overall, when in fact the indicator only shows a decrease in pending cases and no more.  A political inference is required to make the grander case, not a statistical inference.

Because measures can be quickly compiled to suit urgent purposes, ‘data must be handled with care.’  Yet no cautions or warnings adorn the labels of the new Sustainable Development Goals, and we’re seeing a cult of big data thriving and driving the security and justice agenda.

So, what explains the lure of indicators in this field?  It was suggested that all measurement in existing S&J work is organized around the following 5Ps, in ascending order of importance and hope but not frequency, quality, or care.suggested that all measurement in existing S&J work is organized around the following 5Ps, in ascending order of importance and hope but not frequency, quality, or care.

1. Programmes

2. Policies

3. Practitioners

4. Populations

5. Principles.

Todd Foglesong's thesis here was that we want to achieve the big Principles behind S&J (things like equality, human rights or social justice), but we do so in a largely esoteric way and measure it even more indirectly. To begin, S&J actors work actively on the first two areas, Programmes and Policies, in the hope of influencing Practitioners to improve their Populations’ experiences of security and justice: and with it, it is hoped, the realisation of the big Principles. However the measure of that realisation is, according to Todd, no longer the big Principles themselves, but some population’s ‘well-being’. For instance, Eric Posner claims that human rights have failed to improve well-being, and this is an indication of their “ineffectiveness.”  The current culture of indicator development facilitates this kind of estimation.

We were left with some tough questions. Under what conditions do indicators help create introspective organizations, a learning culture in development? Do ever more precise measures of symptoms and problems advance the cause of security and justice? Have we fallen for what Sally Merry calls ‘The Seductions of Quantification’? And, really interestingly, do we need to measure the effects of our own measurements, i.e., do we need indicators for our indicators?