[Stoves] aDALYs 2 (Re: Crispin)

[Nikhil Desai]

Crispin: (changing the thread title, adding 2 because I responded to Ron
earlier, and cc'ing Tami and Neerja since IWA and WBT have come up again,
Ajay and Sumi because they can rectify my quotes on IER, DALYs, aDALYs,
etc. We will take this to WHO unless I am judged to be incompetent.)

You have raised critical policy matters. Mere technical consensus among
those agreeable, or incremental improvements in "knowledge" do not rise to
the level of "actionability". There is a risk that some six years of work
may leave very little actionable material for prudent policies for public
expenditures. But that has been the curse of "better biomass stoves"
before; do not lose heart, for we now know what to dismiss.

Your errors, for the record:

1. "(To make a 'general claim for benefits' is fundamentally different from
claiming 'Joe will live 4.15 years longer if he changes to Stove A and uses
it for >75% of his cooking. ).. The former is easily demonstrated for a
population which is what DALYs are about."

Not true. For two reasons - DALYs are for the dead, aDALYs are for the
living. Any "general" or population-wide reduction in specific disease
incidence cannot be demonstrated. There will be premature deaths and
disability, just blamed on something else, depending on computer algorithms
and developing consensus among 700+ or 70,000+ experts on individual
diseases.

2. "The calculation of DALYs is 'attribution to causes'..'

Not true, again for two reasons. One, it is deaths that are assigned -- or
re-assigned -- causes according to computer algorithms. This is done by
IHME where there are multiple causes of death, no no cause of death listed
or in their view mistakenly assigned ("garbage codes" in death data
included.) Then, once deaths are distributed according to causes, the
causes assigned to "premature deaths" (cut-off age arbitrary) are further
distributed according to "risk factors". This allocation is based on expert
quarrels and consensus, and also how the diseases are clustered. (ALRI or
COPD or CVD are not single diseases, even though a single cause of death is
assigned by IHME). After the aggregate of years of life lost due to death
are computed, they are supplemented by years of life lost due to
disability. Two, death statistics are weak enough as they are in the
developing countries, disability data by disease are spurious; medical
systems are just not geared to such datakeeping. (I have been around some
hospitals in the world.)

3. "Oh, and the text says, by careful wordsmithing, that the WBT is valid,
and is the default test method for everyone, without review, and while the
text permits other methods of testing, they are not valid unless they
produce the same metrics calculated in the same manner."

I do not recall this in the IWA text, but will recheck. If at all, the Lima
Consensus, as Tami has pointed out, was a "place-holder" (or something like
that). WBT was an interim protocol to be tested and revised, and "interim
standards" were to be promulgated under the IWA. The IWA was not reviewed
after three years and I imagine that was because the TC took over. (I don't
know ISO procedures.)

If WBT is regarded as the "default test method", it is a problem for those
who chose to do so and making promises based on WBT. The most significant
actors in this regard are the CDM of UNFCCC and Gold Standard. Neither has
any duty to satisfy the stove user, only have an agreement of agreeable
gentlemen and ladies.

4.  "To make a claim such as a specific health impact from an 'amount of
emissions' requires establishing the metric: emission rate? Maximum
emission rate? Exposure to those rates? Average 24 hr exposure to some
'rate'? You can't leave the details to someone else. When making a claim as
specific as 'this tier stove produces this health response' you need a huge
amount of information that does not yet exist, and even if it did, you
cannot 'calculate' an aDALY using it."

Remember my caution "There is no there there." it is gross error - and
unfair to Kirk Smith, BAMG - to assume that the HAP "Burden of Disease" is
based on any data on emission rates, emission profiles on representative
samples worldwide. All that was used for DALYs is the assumption of
equitoxicity of PM2.5, neglect of PM2.5 other than those from household
cooking fuels, and the Integrated Exposure Response curve. (Cite: Smith's
Millions Dead paper and its Supplement. I can provide my in-text comments
to those who request me independently.)

Much as it may aggrieve you, HAP DALYs have nothing to do with emission
rates or exposures for the Millions Dead, whose lifelong exposure and
disease profiles are unknown. No dose, no response, at least not in terms
of actual measurements.

Have you heard anybody make a specific claim that 'this tier stove produces
this health response' ? That is braggadocio, conceit and deceit. In fact,
the very reason Gates/NIH, etc. are spending hundreds of millions of
dollars on researching the impact of LPG stoves or Philips stoves is
precisely because huge amounts of data are required to make such a link and
even then in small populations, confounding factors make a mockery of
research.

There is no quantified evidence on "health benefits" of "clean cookstoves"
in the West over the last 100 years. Similarly, there is no quantified
evidence that PM2.5 concentration reductions in US over the last 40-70
years (first by fuel bans in certain locations, then technological
controls) have any specific reduction in premature mortality. There are
some correlations, but troubled by confounding factors.

This is why I was aghast at the WHO SFU Guidelines and Emission Rate
Targets. I could accept the % emission reductions that were agreed to in
the Lima Consensus; somewhere along the line, the logic took a flight of
its own without data. The WHO Reviews of Literature led to all this; it is
water under the bridge.

5.  "You have yourself indicated above, twice, that smoking is more harmful
to health than other forms of exposure to smoke, implying that all forms of
PM2.5 are not equal."

But that is precisely what IER is about, of necessity. Burnett, et al.
(2014) wrote:

Although we assumed that the toxicity of PM2.5 exposure, as characterized
by RR, changes with the magnitude of exposure, *we also assumed that at any
fixed exposure level, toxicity is roughly equivalent among all types and
temporal patterns of PM2.5 exposure. These are important assumptions
because estimated PM2.5 exposure throughout the world, whether from ambient
origin or household indoor combustion, has not been differentiated by the
components or sources of fine particulate matter." (RR = Relative Risk) *

and

The key assumptions that underlie the IER, discussed above, largely serve
to justify the integration of risk estimates for different types of PM
exposure. These assumptions, and their tenability, have been
addressed elsewhere (Pope et al. 2009, 2011a, 2011b). U*nfortunately, for
several of the most critical assumptions, those concerning the relative
toxicity per unit mass of PM2.5 of different types (e.g., AAP and AS), not
accounting for the temporal pattern of exposure, and the absence of
interaction among types of combustion, there is little empirical evidence*
against which to evaluate those assumptions or to evaluate in detail
specific implications of their violation. Each warrants additional
research. (Emphasis added.) http://dx.doi.org/10.1289/ehp.1307049 .


Kirk Smith is fond of saying "You get what you inspect, not what you
expect."  There is a perverse application of this dictum, "You get what you
expect if you neglect to inspect, and instead make assumptions buried in
peer-reviewed papers."

Yes, these assumptions render the IER application invalid in my view. But,
as with HAPIT, if you ignore assumptions, you can get away with literal
murder by assumption and saving lives by assumption. You don't need
evidence when assumptions suffice.

6. "There are no 'international standards'. "


You raise a very important legal issue. What is a "standard"? There can be
non-governmental organizations to set a "standard", though they may have no
means of enforcement; market decides. But we have this thing called "Gold
Standard Foundation" which proclaims itself to be a "standards and
certification body". Yesterday at the end of their 57-minute lecture in a
one-hour Webinar, I was able to slip in two questions -- a) What law,
where, acknowledges their role as a standards and certification body, and
b) Who is to certify that their aDALYs are "credible evidence" toward
meeting the UN SDG 3.9.1 which, to my knowledge, makes no mention of aDALYs
for any of the health goals.

I was told I would get an answer by e-mail. I attended the Webinar again
today, and signaled my intent to ask a question (different ones). Abhishek
Goyal (GS) was probably careful enough to declare "We have no more
questions, so let us close".

Even for SDG 3.9.1, the metric is "% of households using solid fuels".
Since the SDG regard solid fuels as by definition "dirty", there is no way
Gold Standard aDALYs for solid fuel stoves of whatever emission rate can
meet the SDG metric. Whatever ISO TC-285 may be doing under WHO guidance to
meet WHO Emission Rate Targets is immaterial.

More than five years of work down the drain. But then, this has happened
several times in the past, as Ken Newcombe (the lead at C-Quest Capital,
collaborator with Goldman Sachs to support the development of Gold Standard
aDALY by BAMG) surely knows. He designed the Ethiopia energy project which
introduced me to the practical consideration of stove and mtad design and
adoption 25 years ago, though by then he had moved on to other projects. He
designed a Nigeria stoves project which he now admits went nowhere for 20+
years. aDALYs are a safe investment bet; so long as enough gullible people
are impressed by glib experts, Gold Standard Foundation pockets the fees
and its cut from the sales of the credits. (Some governments must have
already signed on. They will surely have a proverbial reputational egg on
their faces.)

We have here a situation that the fate of 3 billion people on earth - more
to come in however long it takes to achieve "clean cooking for all" - hangs
on the machinations of a small number of private organizations. Even when
they put their work out in public, nobody seems to read them, so reign of
terror by error rules.

The GACC project on "Evidence Base" is supposed to end four months
according to DfID sources. Maybe its report will help us dispense with
unwarranted assumptions and instead base policy on evidence.

Nikhil






On Wed, Aug 30, 2017 at 6:24 PM, Crispin Pemberton-Pigott <
crispinpigott at outlook.com> wrote:

> Dear Ron
>
> What a delightful set of responses!
>
> There should be lots to digest in a full discussion of the points you
> offer.
>
> >>I feel that the claims being made about what I consider a very technical
> matter of ‎attributing specific health benefits in a calculable manner for
> individuals who adopt a highly improved cooking stove are creating mayhem
> in the world of stoves.
>
>
> *>[RWL1:     I see considerable favorable progress.  All four groups that
> Crispin defines below are doing a good job - on a very difficult task.
> Crispin below complains about more than “ attributing specific health
> benefits”.  Apparently we are to believe that there is absolutely nothing
> useful coming out of a lot of hard work by many talented stove testing
> people. ‎*
>
> I didn't wrote anything of the sort about all the 'hard work'. I am
> wondering right from the start if you are following the conversation
> closely enough. I am pretty sure the phrase 'specific health benefits' is
> self-explanatory. To make a 'general claim for benefits' is fundamentally
> different from claiming 'Joe will live 4.15 years longer if he changes to
> Stove A and uses it for >75% of his cooking.
>
> The former is easily demonstrated for a population which is what DALYs are
> about. There is nothing behind a claim that a particular person will gain a
> particular benefit in future, which is what an aDALY is about. Thanks for
> the opportunity to ‎again clarify that for those who are new to the topic.
>
> >>There are a lot of crooked things done and said about stove performance
> but this business of aDALYs takes the cake. The claims are cooked up out of
> nothing material, substantial, or defensibly true.
>
> *>[RWL2:  If there a “lot” and “crooked”, you’d think there could be a
> cite.  There have been thousands of Physicians involved in the Daly and
> aDaly numbers - for decades.  *
>
> No, they have not. You also are confusing public health with medicine. The
> calculation of DALYs is 'attribution to causes‎' not a medical diagnosis.
> Nikhil covered this well. I said my objections were technical in nature. An
> aDALY is not the inverse of a DALY. It is a completely different thing. If
> a smoker dies in a car accident, and the cause was him leaning over to pull
> out the lighter, you cannot claim that if you convince someone else not to
> start smoking, you have prevented them from dying in a car accident.
>
> *>Surprising that we have such huge international action to discourage
> cigarette smoking considering that health aspects apparently
> were believable to almost everyone by now - using the DALY approach in
> largest part.  *
>
> *The “aDALY” concept is much easier to get at with data on work absences. *
>
> I am pretty sure you do not have a clear idea what an aDALY is.
>
> *>[RWL3:  I recall that the Lima meeting decision was unanimous (I
> don’t wish to take the time to look up that cite, but I gave it on this
> list a while back).   I don’t understand why call anything about stove
> testing to be “gerrymandering”.  *
>
> ‎For those whose first language is not English, gerrymandering is
> manipulating the process by which decisions are made to achieve a
> pre-determined outcome that would not, on balance, have been acceptable
> without the manipulation. When this is done by the convenors and
> administrators of the process, it is sometimes called 'facipulation'
> meaning 'facilitated manipulation‎'. It is also seen when the person
> chairing the process is an advocate, not a neutral party.
>
> The so-called Lima Consensus was written beforehand and sold on the basis
> of creating something temporary as a place-holder for ‎something real, or
> agreeable, or permanent.
>
> 'Something' is not a consensus if it has not been examined and discussed
> openly. For example to accept the WBT calculations and metrics without peer
> review, something I know is close to yours heart, is to not follow due
> process. There was no due process, no review, and the WBT was 'adopted' by
> those who signed, whatever their motive. The decision has no legal basis,
> no scientific credibility and involved a few people the great majority of
> whom had a vested interest in seeing it anointed as somehow a valid and
> useful test method, which it is not.
>
> *>The ISO process is open to every country - and is progressing well
> mostly (I listen in from time to time). India is the site of the next GACC
> meeting and China has plenty of stove conferences.  China was well
> represented at GACC’s meeting in Cambodia.  I still know of no place to go
> to look at the claimed “fundamental errors”.  Why issues of efficiency
> (tiers) and especially CO and particulate emissions should not
> be “relevant" is a mystery (that could be solved with a cite).*
>
> I am not sure what point you are addressing. How did the IWA reflect the
> work done in India and China on test methods between, say, 1982 and 2012?
> The IWA was in great measure written in haste before the meeting, objected
> to prior to the event in the comment period on several fundamental bases,
> suffered through about a day and a half of discussion to yield metrics for
> rating performance that ignored all the lessons from India about efficiency
> calculation, all the contextually of lessons from China, and has three out
> of nine metrics that have no physical basis! It is laughable! Oh, and the
> text says, by careful wordsmithing, that the WBT is valid, and is the
> default test method for everyone, without review, and while the text
> permits other methods of testing, they are not valid unless they produce
> the same metrics calculated in the same manner.
> ‎
> >>I think a thrust in favour of greater consideration of the users has
> come from, among others, Cecil Cook first during the ProBEC era,...
>
> *>[RWL4:  Cecil has reported on this list that he has never asked
> a question to any stove user about possible advantages of char-making
> stoves.  And he didn’t think it appropriate to ask.  Similar to asking
> about the use of cell phones a few years back.  I submit that Cecil should
> not be considered the world’s best expert on how to improve stoves.  And
> probably he has been under instructions.*
>
> I don't see any claim for Cecil being a stove improver. If you ask him how
> to improve stoves he, like Nikhil, will tell you to ask the people who use
> them, and perhaps industrial designers who can bridge the gaps between
> physical stove scientists and the culturally aware intermediaries who have
> observed and understand the complexities of domestic energy use.
>
>
> *[RWL5:  If I as a user or a group want to reduce my fuel consumption -
> I believe efficiency is exactly the right parameter to measure and report.
> And heath issues are also related to the amount of emissions - which is
> known from the tier placement.*
>
> ‎How do you define 'efficiency'? As the WBT does? Or how the TC285 WG1
> does? Or how India does in their national standard? Technical details
> matter.
>
> To make a claim such as a specific health impact from an 'amount of
> emissions' requires establishing the metric: emission rate? Maximum
> emission rate? Exposure to those rates? Average 24 hr exposure to some
> 'rate'? You can't leave the details to someone else. When making a claim as
> specific as 'this tier stove produces this health response' you need a huge
> amount of information that does not yet exist, and even if it did, you
> cannot 'calculate' an aDALY using it.
>
> *>[RWL6:   These tests (ongoing for decades) are comparing stoves for one
> task, not different tasks.  *
>
> In fairness, the task is nearly useless if we speak of the WBT, unless
> people happen to cook that way. ‎If the math involved is defective, or
> there are conceptual errors negative the validity of the metric, the test
> is meaningless. That is why the WBT should have been peer-reviewed by
> experts before it was proposed as a test method in the IWA.
>
> *>Different users and different tasks doesn’t invalidate great work
> by people like Jim Jetter - over many years.  I have seen no one ever
> propose a better single task than heating water.  *
>
> That comment alone indicates the parlous state of stove testing in the
> West.
>
> *>Also,  I believe there is considerable correlation between lab and field
> testing results.  It should be no surprise to anyone that the lab results
> are lower when you watch how little time is spent away from flame tending.*
>
> Do you have a citation for the lab and field test correlation? The ISO
> discussions have frequently mentioned the *lack* of such correlation,
> with many experts noting there is no expectation‎ that such correlation
> should exist. The frequently heard comment that, 'a stove that tests well
> using a WBT tends to do well in the field' is testament to the fact they
> often don't, and certainly do not match performance numbers in the details.
> In short, they may (or may not) be 'generally better' but a Tier claim is a
> specific performance claim, not a general wave of the hand.
>
> If a lab test cannot predict performance in the field, the test is
> inappropriate for rating the expected future performance in use. It means a
> better and more realistic test is needed. I have been saying that for more
> than ten years.
>
> *>[RWL7:   I intentionally mentioned Jim Jetter’s name above because he
> works for EPA.   It is disappointing to see Crispin call EPA measurements
> of pollutants “jazz”.  Sounds eerily reminiscent about arguments on the
> health impacts of cigarette smoking.  *
>
>
> Cigarette smoking is a good example. See above about claiming to have
> prevented someone's death.
>
> *>Jim’s measurements are not on concentrations - they are on emissions.  *
>
> Please reread my post. The WHO emission target is an emission rate which
> is predicted to deliver a concentration. The health argument is based on
> the concentration, which is based on an EPA concentration, which is based
> on assumptions of exposure that automatically come from a concentration,
> and the assumption that all PM2.5 is equally toxic‎. You have yourself
> indicated above, twice, that smoking is more harmful to health than other
> forms of exposure to smoke, implying that all forms of PM2.5 are not equal.
> You disagree with both the EPA and the WHO!
>
> *Others (such as at Berkeley) have done a marvelous job of relating the
> two.   *
>
> I do not agree. They have made 'heroic assumptions' about fuel type, mass
> burned, emissions, concentrations, exposure, inhalation, health impart, and
> attributable mortality. There appears to be no actual data underlying the
> emission rate targets.
>
> *>... The talented folks that I know and respect in going from emissions
> to concentrations clearly know what they are doing - as seen in published
> cites we have mentioned many times on this list.*
>
> How would you know if they know what they are doing? Have you reviewed the
> Single Box Model used by the WHO to calculate the concentrations from the
> emission rate? It is sophomoric in its pretenses. KK Prasad was doing far
> better work at Eindhoven in the early 80's.  ‎What have they been doing for
> the last thirty years?
>
> >>This is a ridiculous situation. We have an entire sector of
> international and national development being bent to suit the careers of
> certain anointed actors who don't wish their preliminary works tossed aside
> because they overreached, or didn't check their math, or avoided peer
> review.
>
>
> *>[RWL8:  This (“anointed actors”) is ad hominem and highly derogatory.
> Again no cites to justify any part of the three allegations.*
>
> I don't see you having any issue with *ad hominem* ‎content. What is your
> point?
>
> >>This nonsense about a 'WHO' approved PM emission rate 'delivering
> countable future health benefits for the user' is ‎intolerable. It is junk
> science.  The objections to it are very technical: concatenated assumptions
> based on several other layers of assumptions cannot make predictions of the
> future health of stove users.
>
>
> *>[RWL9:   This is a mis-statement of what is “approved”.  Tiers are given
> - only ranges on the emission levels that have been released.   *
>
> The emission rates were drafted by a WHO committee. They are therefore WHO
> approved.
>
> The tiers have no physical basis. They are not based on known or
> predictable health outcomes. They are not based on 'emission levels', they
> are 'emission rates'. 'Levels' would refer to concentrations. Please study
> the details.
>
> *>We have tiers because the Lima group knew that nothing firmer could
> be justified at this time.   *
>
> Nothing firmer can be justified at this time. There is basically no data,
> only assumptions. Waiting five years and them make the same assumptions is
> not 'science' or progress.
>
> *>I for one believe that less emission is logically related to health (as
> with cigarette smoke - including second hand smoke).    *
>
> Agreed. It is logically related. Now, put specific numbers on a specific
> change in emissions, and make a valid assertion about how it will affect
> a particular individual's future life expectancy.
>
> You can't. That is the difference between a DALY and an aDALY.
>
> ‎>*T**here is more behind this belief in the harmful inhalation of
> cigarette smoke than WHO use of death and illness statistics.  For
> instance, CSU is even now testing inhalations from stoves with volunteers.
>  *
>
> Why? I can answer that: because no one knows the effects of stove smoke on
> humans, to anything like the extent we know about the statistical
> consequences of tobacco smoking.
>
> >>It happens that at this time, this discussion is critical. The ISO
> Draft International Standard for stove testing is being circulated,
> complete with a set of 'voluntary'‎ emission standards that were forced
> into it over the objections of the experts. Everyone knows that Berkeley,
> GACC, EPA and WHO are behind this - they have been for years. This is not
> new, it is just ‘more'.
>
> *>[RWL10:  This continuing reference to “the experts” is weird.  I know
> quite a few of those participating on the various ISO panels as well as
> Crispin’s “ expert” detractors.  F**rom my perspective, t**here is no
> comparison in either numbers or expertise.   The intent here is to claim
> better expertise from his “the experts”  (un-named) than those many of us
> know from **“**Berkeley, GACC, EPA and WHO” *
>
> I am not sure what you are comparing. Anyone who shows up at ISO meetings
> is given the appellation 'expert'. My intention was to state that the same
> groups who have driven the WBT forward as an 'international test method'
> have done so for years, while avoiding any serious review of it. ‎Papers
> published on the WBT's inner workings universally, without exception, point
> out its many serious shortcomings, the most recent being Riva's analysis of
> the uncertainty of the results, something confirmed by multiple field
> evaluations.
>
> Apparently the GACC feels they are 'past the WBT'. That is good news. They
> join a pretty large crowd living elsewhere in the world.
>
> >...*we are (thankfully) getting closer to an international decision on a
> voluntary standard, so one would be greatly surprised if “not new” and
> “more**” didn’t appear.*
>
> There are no 'international standards'. There are ISO standards that may
> be adopted, modified or not, by national regulatory bodies.
>
> >>Nikhil, who observes much about this and other policy matters, is
> raising the flag of falsity over this abuse of science, trust and position.
> Which hand, at any of those institutions, lifted a finger to sort this
> matter out? ‎No, rather, they perpetuate it carefully, consistently,
> repeatedly, promoting themselves as essential to developing countries and
> their domestic energy policy decision-making processes.
>
> *[RWL11:  Again totally ad hominem - still without a cite (and cites are
> plentiful in the “voluntary” standards backup documents).  Not sure why
> Crispin found it appropriate to put “voluntary” in quotes.   I was equally
> disturbed by Nikhil’s recent flippant reply to Tom Miles - and will get to
> that ASAP.  I deemed this response more critical.*
>
> I am still looking for content in your response. It is clear you are not
> well versed in the critical issues of what is being proposed: that changing
> a stove will deliver a predictable health consequence for the user, and
> that a $ value can be placed on the health change.
>
> That is what I wrote about, and the past problems associated with the
> persistent promotion of methods and claims that do not stand up to
> scrutiny.
>
> >>The bandwagon we should get on is the killing of the WBT, and the
> tradeable aDALY‎. Then we should get on the bandwagon that puts the user
> and their context first. We should do it now. It is quite important.
>
>
> *>[RWL12:  I started by saying I could agree with nothing in Crispin’s
> message.  But I do agree with the last three sentences.   However my
> original blanket objection stands, since Crispin’s first sentence advice is
> 100% opposite from mine.  The WBT IS valuable and should stay
> until replaced with something better - with agreement from other than
> Crispin’s “the experts”.   All “the experts” I trust are advocating
> continued use of the WBT.  *
>
> Can you provide one citation from a published article that explains why
> the calculation methods in the WBT are valid? Can you cite something that
> shows the low power emission rate of CO is dependent on the mass of water
> in the pot? Zhang 2014 proved is doesn not, confirmed independently by Jim
> Jetter at the request of the GACC. This indicates a) the paper got their
> attention, b) that it made an important claim and c) that the claim is
> true.
>
> *>I don’t know the markets where aDALY’s are “tradable” - and doubt one
> exists.*
>
> Don't undermine your status as an ISO TC285 expert. We need all the help
> we can get.
>
> Who here knows the estimated value of an aDALY? It's more than you think.
>
> Regards
> Crispin
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.bioenergylists.org/pipermail/stoves_lists.bioenergylists.org/attachments/20170830/81b6dc7a/attachment.html>