Cost-effectiveness is difficult to establish for a policy ministry
Cost-effectiveness is a measure of how well an organisation uses its inputs to achieve its desired outcomes. For policy agencies, such as the Ministry, demonstrating cost-effectiveness remains a challenge as it is difficult to establish the causal relationship between our outputs and progress towards the long term outcomes; outcomes are frequently impacted by the actions of numerous organisations and individuals. For example, the Ministry has led legislative changes to increase the driving age from 15 to 16. But actual reductions in the road toll will arise through a number of actions by other parties, including education and enforcement of the new driving age.
There are several constraints on the Ministry’s ability to affect transport outcomes. First, the Ministry is only one player in a very large transport system. The Ministry is responsible for $30 million of transport expenditure a year, out of total government transport expenditure of approximately $3 billion a year. Secondly, while the Ministry has an important policy advice role, it does not have service delivery levers with which it can influence transport outcomes. Finally, the Ministry does not have complete control over the work it undertakes. The choices of the government of the day rightly have a significant influence on our annual work programme.
Notwithstanding the above, the Ministry is committed to increasing its overall effectiveness. The question that we need to be able to answer is ‘are we doing the best job that we can with the resources available to us?’ The Ministry addresses this question through its:
- input management
- quality management
- work programme management
- outcomes management.
Staff time, along with their skills, experience and knowledge, is the Ministry’s main resource. We need our staff to value their time and get the most from every hour. To support this approach, we apply project management disciplines to our policy projects. We establish project timeframes and identify milestones that we can manage projects against to ensure projects do not consume more resource than is needed. We are looking to develop new performance management measures for different aspects of our work to enable us to benchmark our performance and then lift our productivity. In the meantime, we can measure how well we deliver our projects on time and to budget.
We manage the quality of our work through our quality of policy advice standards. Each year we have a sample of our policy advice papers independently audited by the New Zealand Institute for Economic Research. This audit not only provides us with an independent view on the quality of our advice and areas that we could focus on for improvement, but it also enables us to see how the quality of our advice compares with other government agencies’. We have also implemented an internal review of all advice papers using the New Zealand Institute for Economic Research criteria. In addition to this, we measure the quality and subsequent cost of our advice using policy advice benchmarks set by the Treasury.
Work programme management
Our annual output plan is a statement of the results that the Minister wants from the Ministry. We manage the Ministry as a single resource to deliver on the whole output plan programme, and we record the actual percentage delivered in our annual reports. In 2011/12 we delivered 90 percent of the key initiatives in the annual work programme.
Delivering on the individual projects in the output plan enables the Ministry to achieve its intended impacts and, through them, make our contribution to the intermediate and long-term outcomes. Our intervention logic sets out the links between our outputs, impacts, intermediate and long-term outcomes.
The Ministry monitors the progress that the whole of the transport sector is making towards the identified outcomes. While our impacts contribute to this, it is not always clear whether the Ministry’s contribution can be singled out from other factors and, even if it could, it is likely to be too costly to undertake. So we use our outcome indicators to focus on the overall progress being made, and use a range of mechanisms to identify opportunities for us to make further contributions where needed.
Key Ministry performance management measures
The key performance management measures that the Ministry will use in 2013/14 are as follows:
- percentage of policy projects managed within timeframes and allocated resources
- percentage of policy project milestones delivered each quarter
- percentage of staff time allocated to the Minister’s priority projects
- 90 percent of regulatory impact statements are assessed as ‘meets’ or ‘partially meets’ the required standards
- 75 percent of policy advice briefings are accepted first time by the Minister
- total cost of an hour of professional staff time devoted to policy advice and other policy unit outputs
- New Zealand Institute of Economic Research’s audit of policy advice papers assesses average score for papers as 7.5 or better
Work programme management
- 95 percent or more of output plan delivered as agreed with the Minister
- 95 percent or more of Ministry impacts achieved through delivering underpinning projects
- results of transport sector progress on intermediate and long-term outcomes published at least annually
- Ministry reviews outcome indicators that show less progress is being made and considers whether further Ministry action is needed.
We will report on these measures annually
The Ministry will bring the results of these measures together in its annual report each year to provide an integrated assessment of its cost-effectiveness and identify areas for improvement in future years.