Evaluation of Road Safety Outcomes to 2005

SUMMARY

Background

1. The "Initial Evaluation ('Stocktake') of Road Safety to 2010 Strategy" was undertaken in November 2004.  The purpose of the stocktake was two-fold:
   
   1. to evaluate road safety interventions funded through the Safety Administration Programme¹; and 
   2. to assess how funding increases in real terms since 2000/01 had impacted on outcomes.  
2. An update of the stocktake was completed in April 2007 to provide further analysis on the outcomes achieved by the implementation of the Road Safety to 2010 Strategy as at 2005, the mid-point of the strategy. 
   
   

   Focus of update

3. The "Evaluation of Road Safety Outcomes to 2005" updates and develops the analysis undertaken in the 2004 stocktake of the Safety Administration Programme".  In accordance with the specifications for the November 2004 report, the focus is on enforcement and compliance measures and the analysis of the interventions is based on the New Zealand Police and Land Transport New Zealand²; road safety output classes as detailed in the annual Safety Administration Programme statements. 
4. In order to assess the effectiveness of the enforcement and compliance measures, the contributions of two other factors to road safety are also considered:
   1. the safety benefits of road engineering projects; and 
   2. vehicle crashworthiness improvement. 

       

   Methodology

5. In Section 2 the report explains the methodology adopted, which is to compare Safety Administration Programme expenditure with intermediate outcomes, namely compliance and enforcement outcomes (e.g. mean vehicle speeds) and then with final outcomes i.e. road fatalities and injuries.  Generally, the analysis compares the average outcome for 1997 to 1999, when funding was constant prior to the start of the Road Safety to 2010 programme, with the outcome for calendar 2005 (or in some cases 2004/05). 

    

   Outcomes

6. The report identifies that the increase in expenditure over the comparison period was around 36 percent in dollar terms, but only around 12.4 percent when adjusted for cost inflation applying to the specific resources financed by the Safety Administration Programme.  The expenditure increase was used to finance a substantial increase in enforcement activity, with the total number of offence notices issued by New Zealand Police increasing by 36 percent.
    
7. Compliance outcomes i.e. speed, drunk driving incidence and use of restraints (seat belts), improved substantially.  Surveys record that the proportion of vehicles exceeding 110kph (i.e. the enforced limit) on the open road fell from around 20 percent to 5 percent and seat belt use increased from 87 percent to 95 percent for adult front seat passengers and from 58 percent to 86 percent for adult rear seat passengers.  The following table indicates that the compliance outcomes for 2005 were better than the targets set for 2004 in the Road Safety to 2010 Strategy.
      

   Table 1: Intermediate Road Safety Outcomes

   Intermediate Outcomes	Ave 97-99	2001	Target 2004	2005	Chg 97/99-05
   Speed, rural winter mean km/hr	101.8	100.2	99	97.1	-4.7
   Speed, % exceeding 110 km/hr	20%	13%	n/a	5%	-15 (% pts)
   Speed, rural winter 85th pctile km/hr	113	109	107	104	-9.0
   Speed, urban mean km/hr	55.6	54.9	55.2	52.4	-3.2
   Speed, % exceeding 60 km/hr	22%	18%	n/a	7%	-15 (% pts)
   Speed, urban 85th pctile	62	61	61	58	-4.0
   Alcohol surveyed % over limit	2.0%	1.2%	n/a	<1%
   Seat belts (adult): front	87%	92%	92%	95%	8 (% pts)
   Seat belts (adult): rear	58%	70%	75%	86%	28 (% pts)
   Children restrained: 0-14, all	88%	89%	90%	94%	6 (% pts)

8. In order to assess the road safety outcomes achieved, the report takes into account two different types of relevant factors: 
   
   
   1. the growth of population, traffic volumes and registered vehicles; and 
   2. the contribution of road engineering safety improvements and vehicle safety improvements. 
9. Section 4 of the report presents outcomes regarding fatalities, injuries³ and social cost of crashes adjusted for each of the growth factors listed in a) above.  Adjusted for the most relevant factor - billion vehicle kilometres travelled (BVKT), the outcomes were:
    
   
   1. fatalities fell by around 30 percent, and 
   2. hospitalisations (of more than one day) and total hospital bed-days fell by around 20 percent.⁴ The following table compares the outcomes with the targets in the Road Safety to 2010 Strategy.
      

   Table 2: Overall Road Safety Outcomes (per BVKT)

   Overall Outcomes (per BVKT)	1998	2001	Target 2004	2005	Chg 98-05	Target 2010
   Rural deaths	18.4	17.4	n/a	13.1	-29%
   Urban deaths	8.68	6.85	n/a	5.88	-32%
   Total deaths	14.2	12.6	9.9	10.1	-29%	6.1
   Drivers killed with excess alcohol	1.81	1.52	n/a	1.44	-20%
   Fatal crashes "too fast for conditions"	3.85	3.41	n/a	2.79	-28%
   Fatalities avoided if belt worn	0.91	0.97	n/a	0.67	-26%
   Hospitalised, more than 1 day	100.9	88.1	66	78.4	-22%	44
   Hospitalised, more than 3 days	67.1	58.9	42	53.5	-20%	28
   Total hospital bed-days	1388	1151	n/a	1061	-24%
   Total Hospitalisations	170	186	140	178	+4%	90

10. Social cost per BVKT was reduced by around 22 percent. While fatalities per BVKT fell 30 percent, the reduction in reported serious injuries per BVKT and particularly in reported minor injuries per BVKT was lower and this is reflected in the social cost analysis⁵. The following table compares the outcome regarding social cost with the targets in the Road Safety to 2010 Strategy.
     

    Table 3: Social Cost Outcomes

    Social Cost (2006 prices)	Ave 97-99	2001	2003	Target 2004	2005	Chg 97/99-05	Target 2010
    Total $M	3847	3235	3380	3290	3324	-13.6%
    $M/BVKT	105.7	89.6	87.1	80	82.7	-21.8%	52.6
    $M/10,000 pop	1.02	0.84	0.84	0.84	0.81	-20.1%	0.78
    $M/10,000 vehicles	1.57	1.23	1.21	1.22	1.10	-30.2%	1.

11. The social cost of road fatalities and injuries is made up of several components. It includes cost of pain and suffering from loss of life and/or life quality, loss of productivity, medical, legal and court and property damage costs. 
12. One major issue regarding the calculation of the total social cost of road crashes is that the total number of reported injuries appear to have been affected by a change in admissions and reporting practices around 2000-01. The following graph illustrates the jump in total reported hospitalisations and reported injuries, which is not matched by an increase in total days stay in hospital, hospitalisations for over 1 day or fatalities.
    
    Chart 1: Road Crash Casualty Data
     
      
    
     

    Attribution

13. Section 5 of the report examines the attribution of the improvement in outcomes between road engineering safety improvements, vehicle crashworthiness improvements and increased road safety enforcement expenditure.  The report recognises that there is no straightforward way to precisely assess what effect the different interventions have had in generating the overall outcome. It considers, however, the implications of independent estimates that have been made of the individual factors⁶ and then assesses the implications.
     
14. The following attribution table provides a summary of the effect of the alternative estimates.  The table takes as it starting point the assumption that all of the observed improvement in deaths, injuries and social cost is attributable to the combined effects of the three factors (road engineering safety improvements and vehicle crashworthiness improvements and increased road safety enforcement expenditure). 
    
    
    1. For each row in the table, the first two columns are the assumptions applied and the third column shows the remaining factor.  
    2. In each row, the effect of the remaining factor is calculated by starting with the observed improvement and removing the effects of the two assumptions to derive the change that remains to be explained and which is attributed to the third (remaining) factor.
    
    

    Table 4: Road Safety Intervention Attribution Analysis

    		Implied Attribution (% Chg per BVKT)
    Assumption 1	Assumption 2	Remaining Factor	Magnitude of Remaining Factor's Effect
    			Rural Deaths	Urban Deaths	Hosp > 1 Day	Total Social Cost
    Vulcan Road	Monash Vehicle	Incr SAP Exp	-7.2%	-11.9%	-0.8%	1.2%
    Incr SAP Exp	Vulcan Road	Vehicle Improv	-15.7%	-19.9%	-10.7%	-8.8%
    Incr SAP Exp	Monash Vehicle	Road Improv	-11.3%	-15.8%	-4.3%	-1.8%

     

15. The results set out in the above table above provide a range of estimates for the attribution to the three factors under consideration of the gains in road safety achieved over the five year period to 2005. The estimates are summarised in the following table.
    
    

    Table 5: Summary of Estimates of the Contributions of Factors

    Factor/Source of Estimate	Vulcan	Monash	Guria	Residual Analysis
    	(2003)	(2005)	(1999)	(This Report)
    Road Improvements
    Rural Deaths	-10.1%			-11.3%
    Urban Deaths	-10.1%			-15.8%
    Hosp > 1 day	-8.4%			-4.3%
    Total Social Cost	-8.4%			-1.8%
    Vehicle Improvements
    Rural Deaths		-14.5%		-15.7%
    Urban Deaths		-14.5%		-19.9%
    Hosp > 1 day		-14.5%		-10.7%
    Total Social Cost		-14.9%		-8.8%
    Increases SAP Expenditure
    Rural Deaths			-5.9%	-7.2%
    Urban Deaths			-5.9%	-11.9%
    Hosp > 1 day			-5.0%	-0.8%
    Total Social Cost			-5.5%	1.2%

    (Note: The assumptions have been imposed in the table that urban and rural deaths change by same % and that hospitalisations >1 day and injuries change by the same %.) 

    Conclusion

16. Along with increased road safety enforcement expenditure, the analysis concludes, on the available information, that road engineering safety improvements and vehicle crashworthiness improvements have made significant contributions to road safety outcomes, particularly in reducing the level of road fatalities and serious injuries over the period 1997/99 to 2005.  
17. The analysis appears to confirm that the increased Safety Administration Programme expenditure under the Road Safety to 2010 Strategy has had a significant effect in reducing fatalities.
     
18. Other key findings of the report are:
     
    1. New Zealand Police increased its activity substantially in the period 2000 to 2005, as evidenced by a 36 percent increase in total number of offence notices for speeding, drink-driving, restraints, unsafe or dangerous driving and other transport offences;
    2. Speed and restraint intermediate outcome targets, as set out in the Road Safety to 2010 Strategy, were met and surpassed;
    3. The percentage of drivers surveyed who were over the legal blood/alcohol limit more than halved between 2000 and 2005;
    4. The major improvements in compliance in (b) and (c) have been achieved in a relatively short period of time;
    5. The 2004 target for reducing road deaths, as set out in the Road Safety to 2010 Strategy, while not met in 2004, was (in effect) achieved in 2005; on a basis of per billion vehicle kilometres travelled (BVKT) road deaths in 2005 (10.1) nearly met the 2004 target of 9.9;
    6. Injury/hospitalisation outcome targets in the period 1998 to 2005 were not met.  Total hospitalisations actually increased by four percent, although it is not known how much of an impact changes in reporting procedures have affected reported injury and hospitalisation statistics.
     
19. The report notes that vehicle crashworthiness improvements appear to provide the largest contribution to improved road safety outcomes over the period.  

    Further work

20. There are two key issues for further work arising from the report:  
    1. The need to clarify, if possible: 
       • The extent to which the increase in the total number of reported injuries (after adjusting for traffic volumes) is a real increase or is the result of changes in practices regarding hospital admissions, ACC claims and Police reporting; and 
       • whether the cost weighting of minor injuries is too high;
       • whether further major speed reductions and restraint gains can be achieved; and 
    2. The need for an ex-post analysis of the safety benefits of road engineering projects and also the actual achieved crashworthiness improvements in the vehicle fleet. Improved confidence in the estimates of the contributions of these factors to road safety outcomes would greatly facilitate assessment of the achievements of the Road Safety to 2010 Strategy.
     
    Download Evaluation of Road Safety Outcomes to 2005 - full report (PDF, v7.0, 339kb) 

    Download Summary of Road Safety Outcomes to 2005 (PDF, v7.0, 89kb)

    ---

    
    Footnotes 
    
    
    1. The Safety Administration Programme funded road safety enforcement activities as part of the Road Safety to 2010 Strategy up to June 2006.
    2. Land Transport New Zealand (Land Transport NZ) was formed in late 2004, by the merger of the Land Transport Safety Authority (LTSA) and Transfund.  Land Transport NZ now undertakes the road safety operational activities previously undertaken by LTSA through the Authority's Land Transport Programme (colloquially known as the Road Policing Programme).
    3. Changes in injury reporting practice, particularly during the period 2000/03, make that data unreliable for assessing changes over time thus forcing a focus on hospitalisation data.
    4. Hospitalisation data is analysed rather than reported injury data because injury data appears to have been affected by changes in reporting practice.
    5. Social Cost estimates rely on the adjustment of reported injury data to take into account the changes in reporting practice and the issues involved are discussed in section 4 of the report.
    6. The contribution of road engineering to reductions in fatalities, injuries and social cost from Transfund projects over the period were estimated in a 2003 study by Peter Vulcan and others.  The contribution of vehicle crashworthiness improvements to reductions in crash fatalities and social cost were estimated in a 2005 Monash University Accident Research Centre study based on newly available data relating to the actual New Zealand crashed vehicle fleet characteristics.  For earlier increases in road safety enforcement activities Guria (1999) estimated an incremental benefit to cost ratio of between 8 and 13 to 1, while Cameron et al (2002) found the Supplementary Road Safety Package a benefit to cost ratio of about 20 to 1.  The report adopts the lower estimate, 8 to 1, in the analysis.