First published in June 2011 at http://conservativehome.blogs.com/localgovernment/2011/06/fact-and-fiction-in-measuring-whether-speed-cameras-save-lives.html

Fact and fiction in measuring whether speed cameras save lives

In 2007 I forced the DfT to admit that their cost effectiveness comparisons of speed cameras and vehicle activated signs were wrong by a factor of ten, though the real ratio was 50. A year or so later then Roads Minister Jim Fitzpatrick admitted that his widely- reported Commons reply that daytime running lights would add 5% to fuel costs was a typing error and should have been 0.5%. He also confirmed that allowing for half our driving being done in the dark and LEDs not tungsten bulbs meant that the real figure is far lower.

I recently wrote to the DfT pointing out that their estimates for the “lost output” of people killed or injured in road accidents was self-evident nonsense, because national output is determined almost entirely by demand, not by availability of labour – especially now. They replied that I “seem to have a point” and confirmed that they are reviewing it, though with no date for a decision. The significance of these figures is of course that they are widely used to assess the viability not only of road safety measures – including camera cost effectiveness – but many other projects not confined to transport.

Another blunder that I first noticed in the camera v signs comparison was the wholly extraordinary way in which the DfT – and for all I know other Government departments – compare projects on the basis of “First Year Rate of Return” – a basis which no commercial organisation in its right mind would use, because it skews decisions in favour of projects with low initial but high recurring costs rather than the opposite, when what really matters is overall long-term costs. The massive recurring costs of camera maintenance and the enforcement system, ignored in the DfT comparison, massively exceed the minimal maintenance costs of signs.

(See other pages on this website for more details of these blunders)

Now we see yet another example of abject failure to understand even basic statistical principles, as Mike Penning, the Road Safety minister, said:

“We want to stop motorists being used as cash cows. For too long information about speed cameras has been hidden in the shadows. These new data will end that by clearly showing whether a camera is saving lives or just making money.”All together now, “Oh no it won’t, it can’t”. Much as I welcome any move to reduce camera numbers, the plain fact is that no Council or anyone else this side of La-La Land has the slightest chance of establishing  from accident data for individual camera sites whether that the influence of any particular camera was benign, malign or neutral, because the small numbers rise or fall not just due to camera effect but also due to:

(a) Long term trends due to a variety of conflicting factors such as active and passive safety features, safer roads, faster emergency response, improving medical skills etc and also, as the BMJ reported in relation to serious injuries, falling reporting levels.

(b) Localised changes such as road or sign improvements or freak weather incidents.

(c) Drivers diverting to avoid cameras, taking their share of accidents elsewhere.

(d) Chance of timing (other things being equal, for example, a nasty accident at 11.50pm on December 31st would result in a fall in the following year, but the same accident at 00.10 on January 1st to a rise – despite no real difference. In the same way data for individual sites based on calendar years or financial year could give quite different results. Across police areas or nationally, these effects will average out, at individual sites they do not.

(e) Regression to the Mean, statisticians’ term for the natural tendency of any variable to move back towards its long term trend after a disturbance. This was ignored by the first annual reports, and paid only lip-service in the fourth, yet is now widely recognised as being responsible for a substantial part of overall observed falls as cameras are almost always installed after a flurry of accidents.

(f) Year-on- year weather variations  (e.g. several mild winter followed by bad winters, poor summers followed by glorious ones).

(g) The state of the economy. Traffic volume levels off or (as recently even falls) in economic crises, leading to fewer accidents. Canadian research shows that this effect is much more marked around the world than simple traffic volumes could explain, and that in recessions worried drivers are more cautious.

(h) Benign camera effect, if any – though it will never be possible to identify an accident that did not happen because a speed camera was present.

(j) Malign camera effect – a matter of record at many sites, including fatalities. Agencies such as TRL and the Highways Agency now admit that some accidents do happen only because a camera was present, though the DfT flatly denied to me any such possibility about four years ago when explaining why a planned investigation had been abandoned.

(k) Random chance. As any experienced driver knows, the difference between a nasty, even fatal, accident and a lucky escape can be as little as a momentary lack of attention on the one hand, or quick reactions and a sharp swerve on the other, or whether or not the feet of a child running across the road are visible beneath the vehicle hiding his body. Incidentally, I narrowly escaped death on the M3 twenty years ago when, in the middle lane, at night, I passed at 60mph an overturned car in the outside lane, facing the wrong way and with no lights. I had not been able to see it because of glare from oncoming traffic. On another occasion I walked across a suburban road in Philadelphia looking the wrong way but escaped with a sprained wrist when I was flung across the road by the slab-sided rear bodywork of the estate car which swerved – not braked – to miss me.

Given all these conflicting factors, many of which are unquantifiable, most of which have never been recorded even if they could have been, and given the minimal amount of data that cannot in any case be statistically significant, the proposition that someone is going to sit down in each area and decide which particular cameras are effective and which, not is self-evidently absurd. How can the DfT and Minister not understand this – in my day any Third Former would have done.

It is also worth noting that many of the 40 or so adverse effects of cameras occur across most of the country, not just on the 3% covered by cameras. Thus even if cameras could be shown – which I very much doubt – to provide net benefit at sites, it is entirely possible that their malign effects elsewhere more than cancel them out.

Another fundamental problem that again the DfT do not appear to recognise is that, by and large, the people being asked to do the analysis are those responsible for camera policy in the past – hardly inclined to admit either to past error or cashing in – and/or whose jobs depend on their continued use. In the majority of the counties where Councils drastically cut or abandoned funding for cameras, they are now paid for  by profits from rapidly expanding numbers of Speed Awareness courses at higher fees under Acpo’s deepy cynical new scheme. As FoI replies confirm, profits are paid directly to police funds and not always ring-fenced to fund cameras.

How on earth did Acpo – a private limited company – get permission for this astonishing  arrangement which, surely for the first time since Robert Peel, gives Police Forces a direct financial interest in imposing as many penalties as they can? For many years they have denied any profit motive, but now it is overt. Will not the glint of a sign in the camera lense not further alienate drivers from the police, and what are the chances of a Force admitting that a particular cash cow is serving no other useful purpose?

So what analysis is actually possible?

i/ Only average results from a large volume of data over at many years can ever be meaningful. (In that context the insistence that data back to 1990 be made available is a step in the right direction – for far too long analysis has been based on comparisons with the three year selection period which is often far from representative.)

ii/ The greatest unknown however is Regression to the Mean. However I am well on the way to completing my analysis of 6m accidents from 1985 to 2007 that separates out RTM from all other factors combined and I expect to be able to show that the accident reductions claimed as camera benefit were almost entirely accounted for by other factors, with no camera in sight.

iii/ What is undeniable is that fatality trends in the speed camera era have been uniquely dreadful, and that the (counter-intuitively at a time of greatly improved vehicle safety) better SI trends turn out, in the view of the BMJ, not to have reflected falling numbers but falling reporting levels.

For all these reasons the arrant nonsense of analysing data for individual camera sites must be stopped, and instead a truly independent and competent analysis – which we have never been given – be arranged. Then all the cameras, not just some, should be switched off.