Are sanctions the key to improve service delivery in social accountability programmes?
In the past decade, a growing set of stakeholders have argued that mixed outcomes in the accountability field are, in large part, the result of weak or inexistent sanctions. It’s commonly argued that if only we could push for harder social sanctions or enforce more severe formal sanctions, we’d see better and more sustainable outcomes. This has been largely unquestioned. It’s time we question it (read the paper here for more).
Andreas Schedler’s famous definition of political accountability is commonly used in social accountability definitions. His focus on ‘punishment,’ ‘sanctions,’ ‘impeachment,’ ‘destruction of reputation through public exposure,’ ‘dismissal’ or ‘legal sanctions’ established a particularly punitive tone in accountability scholarship. Jane Mansbridge’s assessment of the term “accountability” in different languages over the last several decades suggests that accountability has ‘become synonymous with punishment.’
This may be a slight exaggeration, but it’s certainly the case that punishment became a more prominent part of the definition and the conversation more broadly. You only need to look at guidance notes from donors such as the Department for International Development (DFID) and the World Bank who effectively adopted Schedler’s definition, its sequencing, and its hierarchy (see World Bank, 2003; Moore and Teskey, 2006; Malena and McNeil 2010; Grandvionnet et al. 2015).
A decade ago, Anu Joshi argued that without the threat of strong, formal sanctions, transparency and accountability mechanisms are less likely to be effective and less likely to be sustainable. The suggestion here is that social accountability mechanisms will not have impact without strong formal sanctions (Joshi, 2010). Today, there seems to be plenty of evidence to seriously doubt this hypothesis. There are numerous studies with positive effects that lack strong, formal (or social) sanctions. We don’t even have to look outside some of the Randomised Control Trials (RCTs) cited in the most recent systematic review (see Björkman and Svensson, 2009; Alhassan et al. 2016; Gullo et al. 2017).
Nonetheless, I thought it was important to dig deeper. So, Grazieli Zimmer Santos and I reviewed 11 meta-reviews on the social accountability sector, conducted a database search, and consulted 19 experts from academia and practice. This allowed us to select 35 cases to review.
I’ve explained previously in the blog w(h)ither sanctions what we found from the 11 meta-reviews. In short, there wasn’t much empirical evidence that sanctions were anything like as important as scholarship suggests to achieving service delivery outcomes (or for that matter, any form of outcomes). I felt that this was an unsatisfactory conclusion. So, that’s when we decided to search further.
Different mechanisms of change
We carried out a realist review because we found the one Gill Westhorp did on education sector accountability useful and wanted to understand how context might have played a role and were keen to understand potential mechanisms of change.
We found 5 types of mechanisms:
- Sticks: in which service providers and civil servants respond to the actual application of formal sanctions such as fines, pay deductions, or other disciplinary measures (i.e., response to punishment), who are alerted by citizens;
- Big brother is watching: in which service providers and/or civil servants respond in anticipation of the application of formal sanctions from superiors (i.e., fear of being caught and punished), who are alerted by citizens;
- Diagonal accountability: in which service providers and civil servants respond to the threat or imposition of formal sanctions by horizontal accountability agencies such as courts or Supreme Audit Institutions (SAIs), which responded to CSOs (i.e., response to investigation findings);
- Litigation: in which service providers or civil servants respond to legal investigations or lawsuits supported by community paralegals and legal aid organisations (i.e., response to investigation findings), prompted by civil society efforts;
- Naming and shaming: in which service providers and civil servants respond to citizen efforts to name and shame them and change their behaviour as a result (i.e., response to sense of shame).
In many occasions, we found a mixture of these mechanisms in the same interventions. Our background paper has plenty more on these with cases to demonstrate how they fired (or not). For anyone who wants to dig further, that’s the paper I recommend you read.
The positives
We were able to establish a likely link between social or formal sanctions and intermediate effects in half of 35 cases we reviewed with some degree of confidence. Effects included increased service provider awareness and motivation, increased user satisfaction, reduced staff absenteeism, increased instruction time, increased availability of quality textbooks and medicines, lower levels of missing expenditures, improved infrastructure quality, increased funding and staffing made available, and parents increasing investments in their children’s education.
In a handful of cases, we also found impact level changes such as improved test scores and provision of food subsidies. We found that both social and formal sanctions have the potential to contribute to outcomes in various aspects of service delivery in the short term. So, even if there wasn’t much evidence for this in previous evidence reviews, sanctions can contribute to improved service delivery outcomes.
The negatives
However, sanctions-based approaches were rarely sustainable and frequently triggered various negative effects. While each of the aforementioned mechanisms demonstrated the potential to influence service providers’ and civil servants’ behaviour, we found that these results were almost never sustainable (even with a very low bar). This may surprise some of you. But, if you think about it, for how long can you really threaten or impose punishment?
Moreover, imposing social and formal sanctions are not without their costs. We found negative effects (i.e., blowback) in nearly half of the 35 cases we reviewed. Effects included discrediting advocates, threatened loss of funding, reprisals for advocates and whistle-blowers, including court cases and staff being relocated for speaking up, the denial of access to services, an increased concentration of poorly performing staff in some areas, and even threats of violence to MPs and cabinet reshuffles for those who worked with campaigners. So, there are some serious ethical dilemmas here.
Imposing both social and formal sanctions can be seriously counterproductive. Some interventions were said to have damaged staff morale, reducing their attendance, generating conflict among staff and between staff and community members and between staff and patients, damaging trust. Some sanctioning efforts decreased organisational leverage with government counterparts and caused accountability operations to temporarily shut down and decreased budget or contracting transparency. So, there are lots of serious costs to pursuing a sanctions-based approach.
Different strokes for different folks
Another, perhaps surprising, finding was that there are some actors that are regular targets of sanctions, and in many cases, they are a lot weaker than is commonly assumed. Three broad types of actors were the most common targets of punishment: (i) absentee nurses or teachers who had their pay or allowances reduced, (ii) offending officials who were either suspended, relocated, or fired, and (iii) contractors who had to cover the cost of rejected materials or faced lawsuits, alongside civil servants who were involved in contracting.
Closer relationships may perhaps reduce stakeholders’ appetite to impose sanctions. In Niger, Peru, Mozambique, and Nepal, for example, closer relationships created disincentives for confrontation, and in such circumstances a “policing” approach to monitoring was also deemed to be inappropriate and counterproductive.
Conversely, it seems that actors prefer to sanction “others.” That is, when an actor/organisation was outside the group. Short term consultants, contractors, and suppliers were easy (and quite vulnerable) targets for harder sanctions, as we saw in Bangladesh, India, Kenya, the Philippines, Uganda, Colombia, and Indonesia.
The contradictory prospects of hybrids
It’s also been argued that there may be productive combinations of collaborative and confrontational tactics — i.e., hybrids. As mentioned in a previous blog, in the cases reviewed, stakeholders generally did not combine collaborative and confrontational tactics in time and space. Collaborative and confrontational tactics often tend to happen in different locations and at different moments in time. These are often referred to as inside-outside, or hybrid strategies. But, this seems often to be a mischaracterisation.
It’s vital to situate change processes in space and time and indeed to clarify who you collaborate with and who you confront. Fox et al. have recently stressed that both collaboration and confrontation may drive institutional reform. Yet, from our evidence — which partly overlaps with Fox et al., very rarely (if ever) did we find a productive combination in the same time or place, with the same actors.
Fox et al.’s argument often seems to refer to employing different tactics with different state actors (collaborating with some reformers to confront reform opponents). But, various “hybrids” mentioned in recent research also problematically refer to different actions in different places and different moments in time. These are different strategies, not hybrids in any meaningful sense of the word. In the cases we reviewed, when collaborative and confrontational tactics were deployed together in time and space, rather than a virtuous cycle, we found contradictory interaction effects. We unpack this more in the background paper.
As Fung and Wright argued two decades ago, adversarial forms of engagement cannot easily be redeployed for collaborative purposes. From the evidence we reviewed, I believe they are still correct. Thus, as discussed previously, we shouldn’t under-estimate the potential transaction costs of pivots and hybrids.
Soft power can be powerful
Our findings therefore challenge the assumed connection between harder forms of accountability and more significant and sustainable results.
Relatedly, it’s been argued that “weaker” forms of citizen engagement are less effective than ‘strong enforceability.’ We found no evidence to support this contention. A quarter of cases reviewed fit the more collaborative mould at least for a period or in particular places (remember, time and space really matter). And in many cases, those were actually the most successful periods and places. So, soft power can, in fact, be powerful.
This doesn’t necessarily mean that one approach is superior to another. Instead, we argue that in some cases harder forms of accountability may work, and in other cases they may not. We don’t profess universal truths, but with further research we may perhaps find tendencies in certain types of contexts.
Contextual factors
While we were unable to identify strong trends of contextual factors which enabled social and formal sanctions to play a role in enhancing service performance across all cases, we were able to identify a few conditions which we believe offer the greatest promise for further exploration. These conditions were:
(1) supportive leaders who played a role as champions; opening doors or accompanying civil society efforts;
(2) capacity and legal authority of oversight agencies;
(3) competitive elections, which provided windows of opportunity for CSOs to combine political and social accountability efforts, and;
(4) vulnerable public servants and service providers already in relatively precarious situations.
We provide some recommendations in the paper, but the main takeaway is that punishment is not the answer to all the world’s problems. Sanctions can work, sometimes, but they carry considerable risks and can sometimes do more harm than good. So, be careful out there.
You can also see the recording of a presentation I gave on the paper in July for the GPSA here:
