Updated on Oct. 20 - The EPA responded to HPBA’s lawsuit, saying that it is recycling old arguments and using a flawed analysis, and that its lawsuit should be dismissed. Eleven states and one air agency also filed a brief in agreement that provided compelling new data about testing variability and asserted that recent stove certifications are full of errors and deficiencies.
The states’ brief may be influential as it provided recent test data in an arena where very little publicly available data exists. Beyond showing that many stoves can provide repeatable emission data, states say that the ASTM 3053 cord wood method, “an industry-developed test … produces more variability than Method 28R,” the standard EPA crib wood method. The brief also cited recent testing that shows “manufacturers could generate more reliable ASTM 3053 test results by conducting certification tests in a manner that was more representative of consumers’ ordinary use of the stoves.”
We provide a brief analysis and then provide excerpts of key parts of legal briefs by the three parties: EPA, states and HPBA and also the NESCAM memo with variability test data. The briefs are long and parts are technical, but we encourage stakeholders to read these excerpts as they provide important details and context on federal wood heater regulation and the final stages of the much anticipated HPBA lawsuit.
The strong response from the EPA shows that while enforcement has not been a high priority recently, it is heavily invested in maintaining its ability to conduct audits, even though it may have only done so once or twice since the first NSPS in 1988. The 1988 and 2015 NSPS give the EPA the right to retest a stove after it has been certified, known as an audit. However, none of the three briefs ventures to discuss what circumstances could or should give rise to an audit, or whether they would be completely random. More routine audits every 2 - 5 years for the higher volume stove models could be an option.
HPBA is the one litigant in a larger case that has gradually dissipated to one narrow issue: audit testing. The EPA appears to be on solid ground and is backed by many states and non-profit interveners. The data relied on by HPBA is now very old and no manufacturer is offering newer data showing high testing variability.
The variability of emissions testing with pellet stoves or boilers had barely been mentioned in the past. For the first time, data was submitted about pellet stove variability by NESCAUM and cited in the states' brief. AGH believes data like this is important to build greater confidence in and support for pellet technology as a core part of residential renewable heat policy.
These briefs also come on the eve of the EPA sell-through announcement. Stuart Parker, a reporter for Inside EPA
filed a story that noted “many of these same states are also threatening EPA with a lawsuit if it as expected finalizes its sell-through of wood stoves not compliant with step 2 emissions standards. The eleven states closely coordinated their response and highlighted the more notable restrictions that they have enacted to “to mitigate the risks of wood-burning devices.”
Next in the litigation process will be the September 28 filing of briefs by the non-profit intervenors, who include the American Lung Association, Clean Air Council, and Environment and Human Health, Inc. Then, HPBA will have a chance to respond to the EPA and
states by October 9, when they can rebut, if they so choose, the validity of the new testing data submitted by NESCAUM. Only after that
will the DC Circuit Court schedule oral arguments which will almost certainly be virtual. Finally, the three person panel of judges from the DC Circuit
could either be more liberal or conservative
and
may give some indication of what arguments they may be favorable disposed to rule on. The full briefing schedule can be found here.AGH is an independent non-profit that has worked with all stakeholders to advance cleaner and more efficient wood and pellet heaters that can help reduce fossil fuel use and compliment solar panels, heat pumps and other renewable technologies. AGH sided with industry in
supporting a limited sell-through of Step 1 heaters and has sponsored a series of technology competitions to demonstrate innovative solutions to reducing emissions in real world settings. AGH supports the audit provisions in the 2015 NSPS to provide important safeguards to determine potential flaws and lapses in certification testing and better understand emissions from different stove designs in real world settings.
 |
| Simi Bhat, Senior Trial Attorney, DOJ |
Residential wood heaters generate small particles of pollution that can cause difficulty breathing, heart problems, and even premature death. These emissions are so pervasive that EPA estimates that residential wood heaters account for nearly twenty-five percent of cancer risks from toxic air pollution and fifteen percent of noncancer respiratory risks.
After conducting its own analysis on variability and considering other data on newer heaters, EPA concluded that variability was not as high as HPBA supposed, but still present.
HPBA now recycles the same argument about variability, but applies it only to the compliance audit process. Oddly, HPBA ignores its own success—the final standards reflect variability. EPA did not need to adjust the audit provisions because EPA adjusted the standards themselves, both by allowing a margin for variability in the emissions limits and by improving precision in testing.
While HPBA may be concerned that EPA may not administer the audit provisions fairly, this concern is unripe. EPA has not yet conducted an audit under the 2015 Rule. A hypothetical failed audit test by unknown margins with unknown context does not lend itself to judicial review at this time. If EPA revokes certification, that decision can be challenged. This petition should be dismissed.
Residential wood heaters collectively emit hundreds of thousands of tons of particulate matter throughout the country every year. At times and places of high use, wood heaters can contribute over fifty percent of daily particulate matter pollution. Because residential wood heaters are located in residential areas and emit pollution at low heights, even a small amount of pollution from these devices can cause disproportionately high exposure. This exposure is not fleeting because residential wood heaters are often used around the clock.
The manufacturer Woodstock Soapstone submitted comments comparing HPBA’s data to Woodstock Soapstone’s more recent data for three room heaters. Woodstock Soapstone attributed the difference between the variability it observed in test results of its heaters and the variability HPBA described to the “deep[] flaw[s]” in HPBA’s dataset.
Puget Sound concluded that HPBA “disregarded basic questions of data quality and representativeness.” For example, HPBA relied on results from the highest emitting wood heaters, which can have large differences in test results, instead of analyzing the entire dataset. But the variability of the “worst performing” device does not represent the variability across the board. Puget Sound further observed that HPBA’s data reflected “different sampling methods, locations, testing protocols, testing locations, span many years, and do not include data from a balanced or representative cross spectrum of the stoves and independent variables.” Even if HPBA’s dataset were representative and reliable, which Puget Sound did not believe, the real variability was far less than HPBA calculated. Puget Sound discovered that HPBA simply used the wrong statistical measurement to express variability as ± 4.5 to 9 gram/hour. The correct value for the (problematic) dataset of old heaters would be approximately ± 1.5-2 gram/hour.
EPA contracted with the Brookhaven National Laboratory to investigate, among other things, the repeatability of results from hydronic heater tests. Brookhaven tested the heaters by burning wood at three different rates, as specified in the test method for hydronic heaters. At the highest burn rate, results for three replicate tests were within 15% of each other. At the two other burn rates tested, the results were within 3% and 10% of each other. EPA concluded from this study that the “repeatability of cord wood test method” for hydronic heaters can be “very good.” 79 Fed. Reg. at 37,261. Before seeing the results of the Brookhaven study, HPBA had postulated that precision in hydronic heater testing would likely be similar in precision to other wood heater testing
The final 2020 standards are less stringent than initially proposed because EPA incorporated a margin of variability in the standards. EPA concluded that if “precision” in testing “is no better than 1.0 [gram/hour],” the final emissions limit of 2.0 [gram/hour]” for room heaters tested with crib wood would adequately reflect the emissions reductions achieved by stoves that tested at 1.0 gram/hour
The differences between the 1988 and 2015 audit provisions are slight. In the 1988 Rule, EPA required that the audit test be performed at the same laboratory that performed the certification test, until EPA could determine overall test method precision. Test method precision can be broken into two components. There is variability in test results when the same device is tested in the same laboratory multiple times (intralaboratory precision), and there is variability in test results when the same device is tested in different laboratories (interlaboratory precision). Under the 1988 Rule, once EPA determined test method precision, EPA would allow an additional margin equal to only interlaboratory precision when determining whether a device failed the audit test.
SUMMARY EPA did not need to make special allowance for variability in the audit provisions because EPA had already accounted for variability when setting the standards. Even the outdated analysis championed by HPBA does not support the need for any further buffer in the audit provisions.
Manufacturers will have a further opportunity to submit evidence on variability should any of their devices fail an audit test. EPA will consider any relevant evidence manufacturers present during an audit hearing, including evidence on variability. No other mechanism is necessary to address variability in audits.
EPA left the audit process in the 2015 Rule largely unchanged from the original 1988 Rule. Variability has never before been an issue in the implementation of the audit provisions, and HPBA offers no good reason to think it will now.
If EPA revokes certification following a failed audit test, a manufacturer can challenge that decision at that time. HPBA’s challenge to the possibility that EPA might revoke certification based on an unknown test result in the face of unknown rebuttal evidence in a future audit is unripe.
HPBA does not once mention in its brief that the audit process includes the opportunity for manufacturers to present additional evidence. Given that variability is already incorporated into the standards, as discussed above, and that EPA will consider any relevant evidence presented in an audit hearing, EPA did not need to make yet another allowance for variability in the audit provisions.
The only circumstance in which consequences are automatically triggered without a hearing is if a wood heater fails the audit test by 50% or more of the standards. If a device fails so egregiously, then EPA will suspend the certification after giving the manufacturer 72 hours’ notice of the suspension. This suspension may be withdrawn by EPA, and the manufacturer will still have the same opportunity to present any relevant evidence at a subsequent hearing to avoid final revocation.
The 50% exceedance trigger for suspension allows for even greater variability than HPBA itself calculated based on its old and flawed dataset.
2. Excerpts of States brief |
| Nicholas Buttino, Assistant Attorney General for NY |
BRIEF OF STATES OF NEW YORK, ALASKA, CONNECTICUT, ILLINOIS, MARYLAND, MINNESOTA, NEW JERSEY, OREGON, RHODE ISLAND, VERMONT, AND WASHINGTON, AND THE PUGET SOUND CLEAN AIR AGENCY AS AMICI CURIAE IN SUPPORT OF RESPONDENT
The amici States have a compelling interest in protecting their citizens from the emissions of wood-burning devices, which produce multiple pollutants that cause serious health effects. To advance this compelling interest, the amici States have enacted their own measures to mitigate the risks of wood-burning devices, and have participated in efforts to reduce wood-burning device pollutants at the federal level.
The state studies described in this brief confirm that there is no merit to petitioner Hearth, Patio & Barbecue Association’s objections to the audit-testing provisions in the 2015 Rule. As the States’ analyses demonstrate, the audit-testing provisions are necessary to ensure compliance with the Rule’s emissions standards. Petitioner’s testing variability concerns are overblown: those concerns are based on flawed statistical analysis and avoidable testing choices made by the manufacturers themselves. Accordingly, the States urge this Court to uphold the Rule in full, and reject the petitioner’s challenge.
Emerging research shows that long term exposure to PM2.5 increases the mortality rate from COVID-19, including in minority and low-income communities.
The health effects of wood-burning devices are acute in the amici States. The amici States have hundreds of thousands, if not millions, of residents who rely on wood-burning devices for heating. New York alone contains nearly 150,000 homes that use wood as a primary heating source and 500,000 homes that use wood for supplemental heat. In Vermont, 22% of homes use wood as a primary heating source, and 35% of households burn wood for at least some heating.
The amici States agree with EPA that petitioner’s challenges to the Rule are meritless, for the reasons stated in EPA’s brief. The States file this brief to inform the Court of the States’ own research, which underscores that the audit-testing provisions challenged by petitioner are necessary to ensure compliance with emissions standards, and that petitioner exaggerates testing variability as an impediment to reliable auditing.
The Alaska Department of Environmental Conservation has conducted a systematic review of wood-burning devices that have been certified to be compliant with EPA standards. The Department found that 59% of the certifications had inaccurate certification data, and 64% had certifications based on non- representative testing methods—i.e., methods that were not consistent with ordinary consumer use. What is more, three certifications lacked certification test reports altogether. These results suggest serious problems in the certification process. And those problems in the certification process in turn demonstrate why EPA must have a way to check on manufacturers—and hold them accountable—through audits.
The Curkeet study suffers from three main defects. First, the Curkeet study incorrectly applied statistics applicable to a normal distribution to data that was not normally distributed. Second, the Curkeet study improperly divided its dataset and drew conclusions from data that were not representative of the dataset as a whole, allowing particularly extreme values to be highlighted and taken out of context. Finally, the Curkeet study conflated absolute difference in values with a confidence interval, leading to a dramatic overestimate of uncertainty. Based on these problems with the Curkeet study, EPA reasonably discounted Curkeet’s conclusions.
NESCAUM’s analysis shows that testing variability may be minimized if manufacturers use a reliable certification test and scrupulously follow that test method. In issuing the Rule, EPA required labs to follow a certification method designated Method 28R, but allowed for approval of alternative test methods. At the request of manufacturers, EPA approved ASTM 3053, an industry-developed test, as an alternative test method. But using ASTM 3053 produces more variability than Method 28R because ASTM 3053 contains fewer specific instructions. To illustrate, where Method 28R states that manufacturers should use fuel logs that are no less than 5/6 the length of the firebox, ASTM 3053 contains no such limitations.
A NESCAUM study shows the testing variability that may result from ASTM 3053. NESCAUM tested six wood stoves, including five that used ASTM 3053 for their certifications and one that used Method 28R. NESCAUM replicated the results of the stove certified using Method 28R—in other words, there was no substantial testing variability using that method. However, two of the five stoves using ASTM 3053 had high variability between the certification test results and NESCAUM’s results. NESCAUM determined that these stoves used non-typical fueling approaches, such as by burning short logs or logs stacked in unusual ways, which likely explained the variation in test results.
NESCAUM noted that manufacturers could generate more reliable ASTM 3053 test results by conducting certification tests in a manner that was more representative of consumers’ ordinary use of the stoves. For example, the manufacturer of one of the stoves in NESCAUM’s study used the ASTM 3053 test for certification, but with a representative testing technique (by loading the stove as a consumer would). NESCAUM found that that stove emitted almost exactly the amount in NESCAUM’s test that the stove was certified to emit.
3. Excerpts of HPBA Brief |
David Chung, Counsel to HPBA
|
In the 2015 Rule, EPA set very low particulate matter (“PM”) emission standards for wood-fired residential heaters. Not only did EPA require manufacturers to achieve compliance with those low standards during the certification testing process, EPA also promulgated audit testing requirements that allow the Agency to require a manufacturer to retest an EPA-certified appliance either at the same laboratory that conducted the initial certification testing or at another laboratory unilaterally selected by EPA. These requirements are problematic because EPA did not account for the long and well established facts that: (i) the results of repeatedly testing an appliance at a single laboratory often vary significantly (intralaboratory variability); and (ii) there is even greater variability among the results of testing the same appliance at different laboratories (interlaboratory variability). This is generally referred to as “test method imprecision” or “testing imprecision,” but is sometimes also referred to as “test method uncertainty” or “test method variability.”
EPA tried to address test method imprecision in its prior (1988) new source performance standards (“NSPS”) for wood stoves in two ways: promulgating higher emission limits that assumed and accounted for a certain level of intralaboratory variability; and conditionally restricting where and how audit testing could occur, at least until EPA studied and better understood interlaboratory variability. Together, these safeguards helped divide the risk of test method imprecision between EPA and manufacturers, the overwhelming majority of which were—and still are—small businesses. This risk sharing was sensible in light of the inherently variable nature of wood burning and the many potential sources of variability given the complexity of wood heater test methods and the fact that testing can span nearly a full week.
EPA inexplicably proposed to make no such accommodations to account for variability in developing the 2015 Rule. HPBA thus commented at length about testing imprecision, urging the Agency not to require audit testing at all given the demonstrated magnitude of both intralaboratory and interlaboratory imprecision. HPBA also reminded EPA of the need to account for test method imprecision when setting emission standards if the Agency insisted on finalizing audit testing requirements. EPA’s proposal placed manufacturers in the unfair position of not only having to meet very low PM emission limits but having to achieve results that leave a large enough margin below the applicable limits to account for testing imprecision in the event their appliances are audited.
Faced with significant objections to its proposal to effectively ignore testing imprecision, EPA improperly dodged those objections by offering conclusory or illogical responses or (worse) ignoring them altogether. And EPA entirely failed to explain why, in the 2015 Rule, it abandoned the more sensible approach to audit testing and addressing test method imprecision it had codified in the 1988 Rule.
4. Excerpts of the NESCAUM memo |
| Lisa Rector, Policy & Program Director, NESCAUM |
TO: NESCAUM Board of Directors
FROM: Barbara Morin and Lisa Rector, NESCAUM
RE: Reproducibility of Test Results in Step 2 Stoves
DATE: September 4, 2020
Development of new test methods for residential wood heating appliances has been part of Northeast States for Coordinated Air Use Management’s (NESCAUM’s) research agenda for the last eight years. As part of that initiative, NESCAUM purchased and tested several Step-2 certified appliances at contracted research laboratories during the past two years. The primary intent of that research effort was to determine how testing procedures affect emission performance measurements. To establish a foundation for this research, baseline particulate matter (PM) emission performance testing was conducted on the study devices using current certification test procedures. Note that the research lab did not attempt to replicate certification results, nor did it follow the detailed instructions manufacturers provide to the labs, which are sometimes identified in test reports.
The results of the NESCAUM baseline tests are also useful for informing discussions regarding the compliance audit specifications in US EPA’s 2015 New Source Performance Standards (NSPS) for Residential Wood Heaters.
Step-2 Pellet Stoves The NESCAUM study evaluated three Step-2 certified wood pellet stoves with differing heat outputs and pellet delivery systems, identified here as Stoves 11, 13, and 14. The study tests were conducted according to the ASTM 2779 method, which is the Federal Reference Method (FRM) used to certify the compliance of pellet stoves with NSPS emission standards.
The emission results (ERs) measured in all of the pellet stove study runs were lower than 3.0 g/h, the audit test level that would trigger the initiation of revocation procedures. All of the softwood runs were also lower than the 2.0 g/h NSPS emissions limit for Step-2 stoves.
The results indicate that compliance audits are a reasonable requirement for assuring compliance with emission standards for pellet appliances. While ERs were substantially higher in runs with hardwood pellets than with softwood pellets, replicate runs with the same fuel produced reproducible results.
ASTM 2779 does not include specifications about the physical parameters of the pellet fuel used for testing. Because the composition of the pellets has a marked effect on the ERs, it is crucial that test reports include an elemental analysis of the pellets used in the test to enable a fair comparison with audit results. EPA should also consider requiring the use of pellets that are representative of the range of wood pellet types in typical consumer use, including hardwood pellets, in certification tests for these appliances.
Step-2 Cordwood Stoves The NESCAUM research included the assessment of six Step-2 stoves. One of those stoves was certified using the EPA Method 28R (M28R) crib wood test, and the other five stoves using the ASTM 3053 cordwood method.
The ERs measured using the IDC alternative protocol for Stove 7, 12, 15, and 17 would not trigger the actions required when audit testing shows results greater than 150 percent of the standard. As in the baseline testing, however, the IDC ERs for both Stove 9 and 16 were substantially above the certification values and would trigger revocation proceedings. The considerably elevated emissions measured in those stoves, using both the baseline and alternative testing protocols, is evidence that the poor performance is attributable to poor combustion design rather than test method variability and precision issues.
Although the variability in the Stove 16 test runs was within acceptable parameters, the ERs in all runs in that stove were consistently elevated, suggesting that certification values for this appliance are not representative of in-use performance. Stove 9’s run variability was an outlier in the study, and its highly variable performance across testing methods supports the need for replicate testing and compliance audits.
Conclusions on Cordwood Stoves The results of the baseline and alternative testing of cordwood stoves provide evidence that compliance audits are reasonable and necessary for assuring that those appliances conform with emission standards. Baseline and alternative test protocols measured ERs for four stoves was consistently below levels that would trigger revocation actions. The study identified two stoves with substantially elevated and, in one, highly variable emissions that would trigger further compliance activity. Analysis of additional data from the study indicated that elevated or variable emission levels did not result from test method precision or variability issues but instead were an indicator of elevated emissions performance. Problems with the certification testing procedures and operation of those two stoves would likely have been detected by compliance audits.
ConclusionTaken together, the results of NESCAUM’s analysis indicate that the compliance audit requirement is reasonable and would be an effective tool for compliance assurance if fully implemented. The study was able to reproduce certification test results in both pellet and cordwood stoves according to the audit criteria specified in the NSPS, when those certification tests were conducted appropriately. Furthermore, the analysis demonstrated that an audit would likely identify substantial issues associated with unrepresentative operating and fueling procedures and with appliances that have highly variable performance.
The compliance audit requirement in the NSPS rule provides a mechanism for avoiding the inappropriate certification of high-emitting stoves. NESCAUM’s research indicates that replicate testing, such as is used in compliance audits, provides critical compliance assurance procedures necessary to maintain the integrity of the regulatory program. The data also suggest that the emissions measured in audits of well-designed appliances that have been certified using appropriate testing procedures will be acceptable within the tolerances specified in the current 2015 NSPS for Residential Wood Heaters.
