Probation PBR – (a lot) more questions than answers

May 9, 2013 by
Filed under: Public policy 

There should be no one who knows more about the perils and pitfalls of payment by results (PBR) than Justice Secretary Chris Grayling. After all it was he who as a DWP minister oversaw the implementation of the Work Programme which is as yet failing to demonstrate the step change in efficiency and performance that he is hoping to see from the probation service.

There are several reasons to question whether PBR for probation will deliver:

All PBR systems are subject to the inherent tendency for providers to ‘skim’ and ‘park’. The former process involves identifying those in the client group most likely to achieve an outcome (in this case not reoffending) with very little support. The provider then gets paid for something which would have happened anyway without having to add any value (economists call this ‘rent seeking’) and the taxpayer picks up the dead weight cost.

Parking involves limiting the service provided to people who are highly unlikely to meet the outcome and in whom it is therefore cost inefficient for providers to invest resources. The latter phenomenon leads to one of the classic debates in the design of PBR – one which has been rolling around the probation debate – should payment simply be for the result (not re-offending) or should there also be payments for what is often called ‘direction of travel’ – things like attending life skills courses or entering drug treatment. The case for direction of travel payments is that it incentivises providers to do something for the hardest to help; the case against is that it provides scope for gaming and the proliferation of ineffective interventions, such as payments for members of troubled families to attend parenting classes of questionable efficacy.

A particular risk for probation arises from the private/third sector PBR system running in parallel with the state probation service, which will continue to be responsible for more dangerous ex-offenders. The temptation for new providers to dump harder to help clients back on to the probation service will be strong. And while collaboration between providers is essential ,regulating that boundary is likely to be a continual source of tension on the ground.

Given the weak results so far from the Work Programme it is a bold claim that PBR will deliver better outcomes and provide a wholly new service for short stay ex-offenders all within a declining real terms budget.  Some organisations are being very bullish about the possibilities for them to provide great services in the new system, but similar big ambitions articulated by putative Work Programme providers went largely by the board when, under pressure from the Treasury, the contracts were ultimately awarded almost entirely on the basis of price.

Third sector organisations looking to be PBR probation providers should heed the five (yes, five) harsh lessons of the Work Programme.

* Charities, lacking the contract planning and negotiation capacity of the private sector, generally failed to get on the original framework of prime providers. Many wasted tens of thousands of pounds in failed bids.

* Many charities which subsequently entered subcontracting arrangements with primes then felt a lot of pain – in many cases terminal – as the expected and promised client referral rate failed to materialise.

* Relatedly, the PBR mechanism means the contracts have what is sometimes called a ‘smile’ profile. That is, providers get some money up front as an attachment fee but then have to pay for services and make a revenue loss before later in the contract starting to get payments for meeting outcomes. Lacking reserves or access to finance many third sector providers are simply unable to ride out the downswing of the smile.

* The PBR margins on offer to the third sector are anyway very modest on contracts which were already tight even before the prime providers had taken their cut. Many third sector providers are in effect subsidising their provision with charitable income (something which raises difficult issues of propriety and transparency).

* Many third sector providers who thought they were doing a good job found that they were unable to cope with the monitoring and performance requirements of the  various PBR employment  programmes (indeed across the sector third sector providers are having regularly to hand back contacts). Whether this means the system is badly and unfairly designed or that lots of third sector provision is not actually any good is, of course, bitterly contested.

A third problem with PBR as a mechanism for providing public services is the flip side of a claimed advantage. When advocating the Work Programme PBR ministers were apt to talk positively about the ‘black box’ of service delivery. Rather than the over regulated and over managed state services providers would be free to use whatever mechanism works best to achieve the outcome. As one advisor put it to me ‘if getting people to stand on their head and read poetry gets them back into work then what’s the problem’? Fair enough; but the black box in combination with competitive pressure means there is also little incentive for providers to share information, compare practice and improve. ERSA (the trade body for employment programme providers) has been working with Nesta and the RSA to encourage greater information sharing and innovation in the sector but it is not easy to convince providers they should share trade secrets, or even agree a common framework of professional standards.

Most of these issues are generic to PBR as a mechanism. Indeed I have had some rather worrying conversations with civil servants who seem unaware that issues like this are not random risks but core characteristics of such systems, whatever their other advantages. But in probation there is one other major issue.

If a work programme provider fails, someone continues to be unemployed but if probation supervision fails, an ex offender may commit a serious crime. The Work Programme and other employment programmes such as Mandatory Work Activity have been subject to much critical media scrutiny but this is as nothing to what will happen if a badly supervised private sector probation client murders someone. I simply have no idea how a private provider could factor in such massive reputational risks to the cost benefit equation of bidding.

The basic idea of payment by results whereby taxpayers pay for outcomes rather than processes is powerful, especially in a time of austerity, and for a number of reasons PBR is likely to grow. As the chair of the board of a smallish employment service company, I can see both how good practice can deliver powerful results for disadvantaged people but also how tough it is to make money (something which should reassure the public).

The potential upside of PBR makes it all the more important that we understand its inherent pitfalls.



4 Comments on Probation PBR – (a lot) more questions than answers

    [...] the payment-by-results model after the bumpy start of the Work Programme. In a blog post today Matthew Taylor, chief executive of the RSA, sets out “several reasons to question whether PBR for probation will [...]

  1. David Pritchard on Wed, 15th May 2013 5:11 pm
  2. There is a good (economic and ethical) case that skimming and parking is a good thing, and putting disproportionate resources into a service with little chance of success is bad. Whether the providers or the taxpayers benefit from this depends on what tarriffs are set.

  3. Toby Lowe on Fri, 17th May 2013 9:10 am
  4. Nice piece, Matthew. However, you’re only part of the way there. The problems of PBR can’t be solved, and whilst the ‘basic idea’ of PBR is attractive, it is based on an illusion. The core problem with PbR is that it’s not payment for results at all, it’s payment for data production. You don’t get paid for helping solve the complex problems people have (they’re far too difficult to cheaply measure), you get paid for producing the right data.

    Further, you can’t pay for outcomes, because genuine outcomes are the emergent properties of complex systems. Whenever you try to pre-determine an outcome, and then ‘make’ people achieve it (through whatever performance management system you choose) two things happen:

    (1) the ‘outcome’ indicator that is being worked towards is different from impact in people’s lives (because genuine impact is too expensive and time consuming to meaure – so people choose proxy indicators that don’t equate to impact as its felt by those experiencing a service). So people end up working to change the indicator and not helping people with the actual issues they face (so, for example, they force people to get any job, whether that’s the right thing for them or not).

    (2) ‘outcomes’ are too complex to be produced by any given intervention (or combination of interventions). So those who deliver services are being asked to be accountable for things that they can’t possibly be accountable for (e.g, you can’t get someone a job if there are no jobs to be had). This is why ‘gaming’ the system is inherent in PbR – because its the only way for managers of services to generate reliable income.

    All this is well documented – whenever anyone tries to implement such a system, the same results happen. However, because people have viewed these as technical challenges to be fixed, people keep trying (wasting effort and lives in the meantime). We need to realise that such systems can’t be fixed – they’re conceptually flawed. Time for a different approach instead. People are begining to explore these different approaches. Check out:

    and a conference looking at the evidence around Outcomes-Based Performance Management (of which PbR is one example) and exploring alternatives – “Kittens are Evil” – 12th July in Powys, Wales:

    [...] Probation PBR – (a lot) more questions than answers There should be no one who knows more about the… [...]

Tell me what you're thinking...
and oh, if you want a pic to show with your comment, go get a gravatar!