[Stoves] Replication of test results, until you get what you want

[Nikhil Desai]

Crispin:

There is a lot more to your question about variability and uncertainty,
which goes to the core issue -- What are we trying to measure and what is
the predictive ability of the data collected, when the methods themselves
compromise the objective (if there is one)? "Until you get what you want"
is a matter of gaming a regulation; what justifies the regulation in the
first place? (See "Garbage In, Garbage Out" section in an item below on EPA
regulation of residential wood heaters, the subject of your comment from
the Alliance for Green Heat.)

First, is variability -- in fuel quality, operating methods - the same as
"inherent" uncertainty? Or is the distinction only a matter of recognizing
the limits of one's powers of perception? (If I don't care to know, I don't
discover systematic variability.)

For variability, one can in principle set a band of performance instead of
some target efficiency or, in particular, maximum half-hour PM2.5 emission
rate. This may work for heating stoves but not cookstoves.

For heating stoves, the variability in solid fuel quality may be limited in
particular geographies and the combustion pattern can probably be set as
part of the design. For cookstoves, and for the types of cooking I have
observed around the world, these variabilities render IWA-type ratings
rather irrelevant, considering especially that a particular stove may rate
different Tiers for different performance metrics chosen.

Also, assuming, for the sake of argument, that heating stoves are used most
intensively where the heating degree-days are the highest and therefore the
building shell likely to limit ventilation and leakage, you may also get
lower variability in indoor concentration consequences of hourly average
emission rates. Cookstoves when used for cooking alone have vast
variability in building shell and personal mobility. Do not ignore that in
the last 40 years the 2-3 billion users of solid fuel cookstoves changed
their building shells and migrated. There is no basis for assumption that
the same 2-3 billion people have stayed in the same "micro-environments" of
non-occupational use, as Kirk Smith likes to characterize the victims of
PM2.5 premature mortality.)

-------------

How did we get to target emission rates without critically examining the
rationale behind the Tier distinctions and specific levels? There is no
legal imprimatur - at least in the US - for cookstove PM2.5 emission tests
and the definition of targets set has no convincing rationale as a "health
metric".

Remember, TC-285 PM2.5 target emission rates make any sense whatsoever only
by making some heroic assumptions about air circulation, "indoor" and the
subject's mobility, and PM2.5 sources other than the cookstove. (For BAMG,
a human being is only a breathing machine to be modeled, nothing else.
Their aDALY computations make no allowance for variability in humans except
possibly age. Since DALYs are a statistical attribute over a whole
lifetime, age distinctions are useful only when the baseline death
expectancy is valid and fixed.)

And how are we to make sense of these PM2.5 targets? They come from
voluntary "consensus" at Lima and the Hague, and have no authority
whatsoever, just matters of consultant opinions.

The Lima Consensus (2011) listed "POTENTIAL standards associated with the
consensus rating protocol (TO BE DETERMINED AT A LATER DATE)" implying that
a > 90% reduction in PM2.5 emissions will imply a range of 10-20 microgram
per cubic meter emission rate. (I am using capital letters here for added
emphasis. I also see 50 mg/L PM2.5 associated with >90% reduction. The
baseline was Three Stone Fire, without any justification. WHO guideline for
PM2.5 concentration is for all PM2.5, not just from fuels, and to assume
that all PM2.5 exposure is from cooking fuels stretches credulity.)

The IWA 11:2012 took these "potential standards" with % reductions of
course subject to variability in fuel/stove combinations and measured
according to a protocol NOT AGREED TO at Lima Consensus, then translates
the microgram/cubic meter "indoor emission rate" to mg/min rate. This must
have taken some heroic behind-the-door negotiation. (For Tier 4, the IWA
target is less than 2 mg/min, while Johnson-Chiang
<https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4529010/> used 1 mg/min; no
surprise they concluded "To meet WHO emission targets, three-stone fire and
basic charcoal stove usage must be nearly eliminated to achieve the
particulate matter target (< 1-3 hours per week)."

IWA is only guidelines <https://www.iso.org/standard/61975.html>, not a
standard, and even the reference to a regulatory standard for measuring
 gas flows, CO and PM2.5 emission rate is NOT applicable to cookstoves.
(The IWA reference is US 40CFR60, Appendix A, Methods 1-2, 5A and 10. Did
nobody bother to check that the regulatory authority for these is for new
stationary combustion sources generally, but not for cookstoves? EPA has
never had any authority to regulate cookstoves within US boundaries, nor
has it engaged in any research on cookstoves pollution and health in the
US.)

In real life experience of cooking, you get not only the variability in
fuel and operating practices but also emissions from foods and other
sources but also exposures in- and outdoor, plus, most importantly,
variability in location and exposure, daily and seasonal, with practically
no means of measuring real-time exposures except small batches of, um,
subjects.

There is no "cause-effect" theory for per minute PM2.5 emission rates from
cooking fuels and DALYs for any particular cohort. The WHO literature
reviews did not examine each of the epidemiology for context and relevance,
just passing on "meta analysis" of published studies. (Who is to say that
with all the uncertainties and variabilities in study designs and cohorts,
the truth may lie not in what is published but what is not? This is the
fatal flaw of peer-sanctioned knowledge that is not verified in actual
practice. Don't forget, there is no study of cookstove emissions and health
in the rich countries; what EPA did beginning with PCIA is merely an
employment program for staff and consultants.)

So why are we even taking "indoor emission rate"? What is an "indoor
emission rate" -- emissions from cookstove fuels in an area that has a roof
and four walls, with total volume of 50 cubic meter and no ventilation, or
with two windows and a door?

The definitions, the methods, and the targets are all suspect. Ignoring
variability and uncertainty - and not having a credible theory based on
legally sanctioned methods - together amounts to intellectual smoke and
mirror.

As for the only instance where stationary wood stoves have come under EPA
New Source Performance Standards purview, namely residential heating
stoves, read a commentary It is time to face the truth about EPA's Standard
of Performance for New Residential Wood Heaters
<http://www.hearthandhome.com/magazine/2017-04-25/straight_talk.html>
(James E. Houck, Straight Talk, Hearth and Home, May 2017).

*Garbage In, Garbage Out*: "Bottom line is that uncertainties inherent in
needed measurements combined with the variability in the operation of the
wood heater do not allow for a very accurate determination of a
certification value. .. It was clear that science took second seat to make
a more restrictive standard even though it was inherently inconsistent with
the facts."

Who is to say that the TC 285 standards and test protocols will not be
another instance of the same? Kirk Smith already warned in 2014 that the
WHO-authored norms will get tighter over time. There is no justification
for that threat, and in any case, I doubt EPA - with a sorry history of its
own effort in regulating residential wood heaters, and no authority over
cookstoves in its own geographical jurisdiction (the US and some
transboundary pollution issues) - is a good guide for what biomass stovers
are trying to do.

I see no point in technical competition to chase WHO-set "health metric".
Except Garbage In, Garbage Out. TC-285 members should probably rethink
their interests and methods.

Nikhil




------------------------------------------------------------------------
Nikhil Desai
(US +1) 202 568 5831 <(202)%20568-5831>
*Skype: nikhildesai888*


On Wed, Jun 28, 2017 at 9:41 AM, Crispin Pemberton-Pigott <
crispinpigott at outlook.com> wrote:

> Dear Friends
>
> There is an interesting post and brief comments at the Alliance for Green
> Heat here
>
> https://www.facebook.com/AllianceforGreenHeat/photos/a.12883
> 5288983.103745.120546153983/10155593812988984/?type=3&theater
>
> The question addressed is why so many stoves are tested and get a PM
> emission rate of 4.4 grams per hour, just under the 4.5 g/hr EPA limit.
>
> [cid:image001.png at 01D2F056.2EC67F20]
>
> One of the answers is that with performance so variable, if one gets some
> results above the 4.5 limit, just keep re-testing until some of the results
> are below the limit, and accept those.
>
> This raises an interesting question about what to do with the variability
> of test results, caused either by an inherent variability of the stove
> performance or the inability of the equipment to measure performance. There
> are different approaches taken by different countries on this point.
>
> The issue is uncertainty. As has been discussed in detail by Fabio Riva in
> a couple of papers so far, the uncertainty of test results limits one's
> ability to make performance claims. Well, you can make a claim, but how
> uncertain is that claim?
>
> A point made in the comments is that once manufacturers have a result that
> 'meets the 4.5 g limit' they are not interested in testing more to obtain a
> lower result. If the target were lowered to 4.2 additional testing would
> eventually yield a set of 'acceptable' results with an average of 4.1.
>
> What is the variability in results? If you use % of value, Norbert Senf
> says in the comments it is ±35%. Interestingly, as the stoves get cleaner,
> the variability is larger and larger. This has implications for 'tiers of
> performance' because placing a stove on a tier requires one to claim that
> it is on one tier and not one or two others. The 'tighter' the performance
> requirement, the less clear it is that the target has been met.
>
> Regards
> Crispin
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.bioenergylists.org/pipermail/stoves_lists.bioenergylists.org/attachments/20170628/369e6776/attachment.html>