Prepared for: Ministry of Transport, PO Box 3175, Wellington

Author: A. Campbell, Fuel Technology Ltd

Other Principals: J. Gething, R. Raine, S. Elder and K. Jones, Energy and Fuels Research Unit, Auckland UniServices Ltd

Date: 30 June 2006


Some conclusions drawn from the Pilot work have been based on relatively small vehicle samples. Due to the small sample sizes, care is required in extrapolating these findings to the New Zealand fleet, as the results are indicative only unless supported by appropriate statistical analysis.

1. Executive Summary

1.1. Introduction

This report concerns the trial of simple emissions testing of diesel vehicles that was carried out in late 2004, known as the Vehicle Emissions Pilot Project (the Pilot). The report follows on from the report (Campbell et al) Vehicle Emissions Pilot Project - Petrol Vehicles, which detailed the trial work and analysis carried out for simple emissions testing of petrol vehicles. These trials were instigated in response to the Government's decision at the time to introduce a simple emissions testing regime, specifically the idle simple test for petrol vehicles and the snap acceleration test for diesel vehicles. Experiences and analysis from the trials were intended to support the development of this testing regime.

The work carried out for the Pilot comprised: appraising the indicative current performance of the vehicle fleet with respect to simple emissions by testing a sample of vehicles using simple emissions test methods; piloting simple emissions testing in order to gain the experience which would be needed in the development of a simple emissions test regime for New Zealand; and developing an understanding of the improvements in vehicle performance that might arise from the introduction of such a regime. The findings from this work are summarised below.

1.2. Simple Emissions Performance of the Diesel Fleet

Snap acceleration testing was trialled at 13 sites around New Zealand. The sites chosen represented a range of different test site options, from sites where only warrant of fitness inspections were otherwise carried out, to vehicle repair workshops. Testers of a variety of different backgrounds and competency were also involved in the trial.

The Pilot tested around 800 diesel vehicles to the Pilot-developed snap acceleration test procedure, a procedure very close to that used in the UK. Emissions profiling analysis was conducted on data from a smaller sample of vehicles (197 light diesel vehicles and 255 heavy diesel vehicles), the smaller set the result of screening the field data by various quality assurance tests, among others. Analysis techniques used included multiple variable regression analysis using Statistical Analysis Software (SAS), a method by which many variables can be considered at the same time instead of confining analysis to two- or three-dimensional comparisons.

The emissions performance profiling analysis found:

  • there is a high degree of variability in the snap acceleration results from diesel vehicles
  • the variability in snap acceleration results for the sample analysed was best described by a model using the four variables, engine technology, secondary performance indicator (SPI, a variable calculated from odometer and year of manufacture), visible smoke (as judged by the tester by sight) and gross vehicle mass (GVM). The factors used in the model indicated a trend towards lower snap acceleration results for: more advanced engine technology; lower distance travelled; more recent year of manufacture; lower visible emissions; and higher GVM. This model had a coefficient of determination (R2) of 0.24; that is, only 24% of the variability in snap acceleration results could be described by a model using these four variables. In this model all variables were statistically significant above the 95% confidence level (that is, p-value <0.05). The significance of visual smoke in this model provides some support for the use of simple visual checking of vehicles as a possible screen for emissions testing vehicles (rather than metered snap acceleration testing)
  • the emissions performance profiling analysis considered engine technology described in three simple ways: (simple) non-turbocharged; (simple) turbocharged; and turbocharged plus oxidation catalyst (advanced technology), which was a kind of shorthand description of major advances made in diesel engine design. For the sample analysed, engine technology was found to be a statistically significant variable, although only between simple and advanced engine technologies, with no statistically significant difference between the two simple engine technologies. Note that the data sample analysed only contained four heavy diesel vehicles and 13 light diesel vehicles that also had advanced engine technology, and these were all relatively recent year of manufacture. It would therefore be difficult to draw strong conclusions from this particular analysis, although the results are in line with the emissions performance expected when a practical, technical appraisal of the design of the technology involved is conducted. This strengthens this, otherwise weak, conclusion
  • vehicle origin was found to be a statistically significant variable when considered by itself: that is, without also considering other variables at the same time. However, when engine technology and year of manufacture were also considered, vehicle origin was no longer statistically significant indicating that vehicle origin was very weak in describing the variability in snap acceleration results
  • 'percentile plots' were created, plotting vehicles in order of increasing snap acceleration result, allowing the proportion of vehicles failing to meet given cutpoints1 to be determined. 23% of light vehicles and 12% of heavy vehicles in the analysis data set did not meet a cutpoint set of K=2.5m-1 for non-turbocharged vehicles and K=3.0m-1 for turbocharged vehicles (a cutpoint set which is in use in the UK, and is based in turn on the requirements for Europe)
  • the average snap acceleration performance for Used-Japanese vehicles entering the fleet was found to similar to marginally better to the average for the existing fleet. At this level of performance, their entry to the fleet produces no significant improvement. New vehicles, by contrast — or, at least, vehicles of more recent manufacture, which are more likely to feature more advanced engine technologies — are expected to perform better than the existing fleet average, and their entry would produce an overall improvement in fleet performance
  • engines designed in the 1960s and before, and some of more recent design, may exhibit elevated snap acceleration results even when in good condition, and some allowance would need to be made for these vehicles should an in-service snap acceleration testing regime be introduced to New Zealand
  • diesel vehicles fitted with oxidation catalysts are beginning to enter the fleet. Removal of a functioning oxidation catalyst from a diesel vehicle is not expected to change the snap acceleration result significantly (but is expected to affect the emission of other species)

1.3. Snap Acceleration Testing as an Indication of On-Road Emissions Performance

The Pilot tested 39 light vehicles and 80 heavy vehicles both over vehicle dynamometer drive cycles (used to indicate on-road emissions performance) and to the snap acceleration test. The comparison of drive cycle emissions results with the snap acceleration results found there to be a poor relationship between the two for the emission species measured (PM, NOx , HC and CO, in the case of drive cycle tests) and for a range of test cycles.
As an example, the comparison of the IM240 PM measurement versus the snap acceleration result for light diesel vehicles provided a coefficient of determination (R2) of 0.38: that is, the snap acceleration result describes 38% of the variability in the expected on-road PM result as given by the response to the IM240 test cycle. Simply put, while a positive trend exists, whereby a vehicle with a high snap acceleration is more likely to emit higher PM in on-road driving, overall the snap acceleration test is a poor predictor of on-road emissions performance - the snap acceleration test does not reliably tell us whether a vehicle is a low or high emitter of PM in on-road operation.

1.4. Emissions-Related Repair

The evaluation of the emissions-related repair of diesel vehicles included: the analysis of data from the snap acceleration testing of 27 vehicles before and after repair; consideration of three vehicles dynamometer tested before and after repair; consideration of various international papers on the subject, and consideration of information from the industry provided during the Pilot. The principal findings were:

  • emissions-related repair is expected to lower the snap acceleration result and the average reduction in the result is expected to be significant for vehicles exhibiting high levels of visible smoke emission (of the order of a K=3.0m-1 reduction, on average)
  • on average, repair of diesel vehicles exhibiting high visual smoke emission is expected to decrease PM emission and increase NOx emission. Any change in fuel consumption is expected to be small to negligible
  • a high proportion of the repairs of vehicles exhibiting high levels of smoke emission is expected to include servicing of the injectors. Blocked air filters and pumps that are not correctly calibrated are also common faults. A blocked air filter is less likely to be the cause of high visual emissions by itself
  • the range of costs for the repair of a diesel vehicle exhibiting high levels of visual smoke emissions is from around $150 for a simple injector service to many thousands of dollars for a major overhaul or the replacement of an engine
  • the snap acceleration test was not that useful for fault diagnosis other than as a simple check of general visible smoke emission. Even then, the acceleration test does not need to be performed to a stringent test procedure to provide near its full worth for diagnosis - the engine either produces high smoke emissions or it does not
  • the vehicle repair industry in New Zealand appears sufficiently tooled and skilled for the repair of diesel vehicles, including those vehicles fitted with more advanced engine technologies
  • the replacement of cambelts is believed to have been deferred on many light vehicles and, should snap acceleration testing be introduced across the fleet, then it is likely many vehicles will require cambelt replacements

1.5. Implementation of Snap Acceleration Testing

The implementation of snap acceleration testing in New Zealand was considered through analysis of all the information gathered during the Pilot, from experiences during field testing to the detailed information gained through laboratory testing of various snap acceleration procedures. The major conclusion of this analysis is that snap acceleration testing is not recommended for New Zealand as the basis of a mainstream vehicle emissions control programme. The main factors on which this conclusion was based were:

  • around one-quarter of the fleet were not built to any emissions standard and it may be difficult to require these vehicles retrospectively to meet a given emissions performance standard, unless it were a very lenient pass-fail cutpoint
  • the poor relationship between snap acceleration results and on-road emissions means that there is a risk the results of snap acceleration testing would be challenged
  • implementation of snap acceleration testing is expected to be relatively expensive and there is a risk that the industry would over-invest in the initial years of the regime
  • the snap acceleration test has limited applicability for the modern vehicles now entering the fleet
  • there was a good correlation between the snap acceleration result as given by a smoke meter and that as judged by eyesight by the tester. A visual test may be more appropriate for New Zealand in the short term.

Nonetheless, snap acceleration testing may be useful for awareness purposes, for the emissions testing of specific targeted vehicles, or in support of other vehicle emissions programmes. For example, the snap acceleration testing of used imports before they are permitted to enter the fleet for the first time is recommended. This might be replaced by a more reliable short-test indicator of on-road emissions performance, should such an appropriate short test be found in the future. Note that used diesel vehicles now entering the fleet would have been designed to meet an acceleration test and the introduction of meeting a snap acceleration requirement should therefore be relatively straightforward from a compliance point of view. On the other hand, the development of a regime based on an alternative short test may be a protracted process, as arguments may arise over the suitability of any test which vehicles have not been specifically designed to meet, even if it is a test that they would pass if they were in good condition.

There is currently no mechanism to demand the repair of a high-emitting diesel vehicle unless it emits continuous visible emissions. This less-than-satisfactory situation will persist unless a snap acceleration test regime or high-emitter test and cutpoint of some sort is introduced. This weakens the authority upon which other emissions reduction programmes could be supported.

Alternatives to the snap acceleration test have been suggested. These include: visual inspection for visible emissions at the time of safety inspection; a mechanism to forbid, or at least discourage, tampering with emissions-related equipment; introducing a minimum emissions build for vehicles entering the fleet for the first time; broadened enforcement of the 10-second Rule; and (as has been mentioned) the snap acceleration testing — or a more robust check of emissions performance — of used imported vehicles before their entry to the fleet. Note that the use of remote sensing to detect high emitting vehicles has not been included in this range of suggestions, as it is unlikely to provide a reliable indication of emissions performance for diesel vehicles unless there is strict control over how a vehicle is operated at the time of sensing.

Should snap acceleration testing be introduced, a recommended test procedure for New Zealand has been identified. This includes the provision of a 'fast pass' option to dispatch vehicles showing very low emissions quickly. Such a snap acceleration test regime would require a number of supporting systems, including:

  • a Standard or Code of Practice for snap acceleration testing, including the specification of smoke meters
  • a minimum proficiency standard for testers
  • a quality control programme to manage the maintenance and calibration of smoke meters, including an accreditation system for laboratories and technicians performing this work
  • a quality control system to monitor test site performance, with the ability to intervene where necessary

It is expected that to support the testing and repair work required by a snap acceleration programme involving two-yearly testing of vehicles manufactured between 1985 and 2000 (a scenario developed for Pilot analysis purposes), the industry would require the addition of at least 300 full-time personnel or their equivalent. The introduction of snap acceleration testing would require careful management, as this step increase in industry capacity would take several years to achieve, at best, and also risk the industry over-investing in the earlier years. An over-optimistic introduction would also risk the quality of the programme being compromised.

Once introduced, a snap acceleration test would be expected to take 5 to 20 minutes and cost around $33 on average, ranging from $20 to $56 depending upon the facility type and whether vehicles may be tested easily. Higher costs would be expected during the regime start-up period.

The snap acceleration test is expected to be difficult to integrate into an existing safety inspection without extending the duration of the inspection, and flexibility must be allowed as to how these two systems are integrated.

1 A result above which vehicles are considered to have failed the test.