[Stoves] aDALYs 2 (Re: Crispin)

Fri Sep 1 19:56:18 CDT 2017

Dear Nikhil

Thank you for your detailed responses. You have in-filled what I was reluntant to add to an already long post.

>Crispin: (changing the thread title, adding 2 because I responded to Ron earlier, and cc'ing Tami and Neerja since IWA and WBT have come up again, Ajay and Sumi because they can rectify my quotes on IER, DALYs, aDALYs, etc. We will take this to WHO unless I am judged to be incompetent.)

Understood and I am replying to same.

>You have raised critical policy matters. Mere technical consensus among those agreeable, or incremental improvements in "knowledge" do not rise to the level of "actionability".

It is this which concerns me at this time. I have invested enough time in worrying about what was conceptually and mathematically defective. As the stove sector gains visibility, it is important that it gains commensurate credibility. In my view that is being compromised by legacy agendas to ‘lead’ and ‘direct’. I work and volunteer for organisations that are at substantial institutional reputational risk if they stumble on their first entry into the arena of domestic energy at the individual level. That is why policy matters matter.

>Your errors, for the record:
>>1. "(To make a 'general claim for benefits' is fundamentally different from claiming 'Joe will live 4.15 years longer if he changes to Stove A and uses it for >75% of his cooking. ).. The former is easily demonstrated for a population which is what DALYs are about."

>Not true. For two reasons - DALYs are for the dead, aDALYs are for the living. Any "general" or population-wide reduction in specific disease incidence cannot be demonstrated. There will be premature deaths and disability, just blamed on something else, depending on computer algorithms and developing consensus among 700+ or 70,000+ experts on individual diseases.
I think you made this clear before. The specific point I was trying to emphasize to Ron is the difference between making a generally true observation (breathing smoke from fires is bad for you) and making a specific claim with a numerical value attached as to the future outcome of an intervention taken today.
DALYs are about populations in general and guessing wrongly a bit there and there misinforms health policy a bit, and money is (in general) not changing hands on the basis of it. aDALYs are being proposed as a funding mechanism with ‘tiers’ of performance achievement (however defined and set) being tied to ‘bankable’ future outcomes. In the latter case, the stoves are being marketed institutionally, as you have correctly described it, as ‘pills’. Take one per day per family and it will have effect ‘X’.
The model for this financing has reached such a refined stage that there is an emphasis on determining what % of the time a stove is being used so that the $ value of the stove can be discounted for ‘stacking’ it with another stove or 2 or 3 of lesser medical value. This level of intellectual fiction should not be allowed to permeate the Donor space.
>2. "The calculation of DALYs is 'attribution to causes'..'

>Not true, again for two reasons. One, it is deaths that are assigned -- or re-assigned -- causes according to computer algorithms. This is done by IHME where there are multiple causes of death, no no cause of death listed or in their view mistakenly assigned ("garbage codes" in death data included.)

Well, in fairness I was indicating ‘causes’ per death with an apportioned contribution based on a population cohort. I don’t have a comment to make on the assignment of all deaths to a single cause for some abstract reason. Obviously such an exercise is fraught with potential to misinform the policy makers (i.e. to lie).

>>3. "Oh, and the text says, by careful wordsmithing, that the WBT is valid, and is the default test method for everyone, without review, and while the text permits other methods of testing, they are not valid unless they produce the same metrics calculated in the same manner."

>I do not recall this in the IWA text, but will recheck.

Here is the first substantial line in the IWA 2011:12, following the Index and list of participants:

[cid:image001.jpg at 01D323C9.81087490]

It plainly says that the WBT is a valid test method agreed by the participants. Please note that there was no proposal to the group asking whether or not this was an acceptable statement for inclusion in the resolutions, meaning the truth of the assertion was not debated, and no presentation or documentary evidence supporting this assertion was made. All other test methods, whether referenced in the document like the SeTAR HTP, would have to be reviewed by a committee which the IWA-forming group had not authority to constitute, and for which there is no process for homologation.

It is an unsupportable assertion that the WBT provides the outputs needed to match the requirements of the IWA performance targets.

In fact, only one of the 10 metrics in the IWA was produced by WBT 4.1.2. This is appalling. How can a test method that is missing 9 of the 10 requirements of the IWA be ‘valid’? This is quite apart from its internal problems. It simply did not produce the outputs needed to compare the performance against the IWA’s requirements!  How could the situation possibly be worse?

There is only one approved test method, it doesn’t give the answers needed, and there is no process to validate a changed version of the WBT or any other test method. In spite of this, a group of actors persisted in maintaining the fiction that all was well and we can start distributing stoves and exchanging money based on the results of the WBT.

So serious was this failure that the GACC, having officially adopted the WBT as their product (at the Carolina conference) started editing the calculations to correct some of the most obvious errors and changing or adding the output metrics, calling by the names that appeared in the IWA text. No public process of review of the test method or the changes was conducted. In fact, I found that on at least two occasions, (April and June, possibly twice in June) the spreadsheet was edited and re-uploaded to the GACC website (the official source of the sheet) without notifying the user that any changes had been made, and posted using the same version number (4.1.2).

I raise my concern about making such anonymous, undocumented changes to the only approved test method, and it was denied by Ranyee Chang who said that this had not happened. Some on this list may recall that interchange. I provided clear evidence that this had indeed occurred. I also found the name of the person who was making the changes. To my knowledge no peer review of the alterations was undertaken.

This should be evidence enough that the IWA anointed the WBT 4.1.2 as the only test method to be used when making an assessment of a stove’s performance when checking it against the IWA tiers of performance.

A serious attempt was made to retain this ‘set’ as an ISO standard (the tiers and the WBT).

>If at all, the Lima Consensus, as Tami has pointed out, was a "place-holder" (or something like that). WBT was an interim protocol to be tested and revised, and "interim standards" were to be promulgated under the IWA. The IWA was not reviewed after three years and I imagine that was because the TC took over. (I don't know ISO procedures.)

I think it was not renewed because everyone was starkly aware that it would not survive an independent expert review. It was enough to say, keep using it while we wait for the ISO process to be completed. That is exactly what happened. We will see when it is published how well the ISO experts did at providing a validated, externally and expertly reviewed test method. Ranyee has already indicated in a message to this list that the ISO process constitutes, by itself, ‘expert review’.  As we have already discussed in recent days, an ‘expert’ at the ISO is anyone who joins the process.  My definition of external expert review differs from hers.

>If WBT is regarded as the "default test method", it is a problem for those who chose to do so and making promises based on WBT. The most significant actors in this regard are the CDM of UNFCCC and Gold Standard. Neither has any duty to satisfy the stove user, only have an agreement of agreeable gentlemen and ladies.

This is the 800 pound gorilla in the room. A performance rating based on the WBT (see the CDM documentation and GS methods for details confirming that they do indeed require, or hint that they require the WBT) is sure to mislead the buyer of traded carbon offsets (CERs etc) for the very reason Ron says he likes it: It frequently, for certain stoves types, rates as “fuel-efficient” stoves that user more fuel than the baseline product they replace. The direct cause is the pretense that charcoal produced from wood consumed is ‘unburned fuel’. Any organisation that trades in carbon-dioxide emissions is at reputational risk if the WBT was used in the performance evaluation.

This risk unfortunately extends to the use of the CCT, another uncorrected, un-reviewed test method with no custodian. While there might by some reputational protection for those using the KPT, which is a field observation evaluation, it is being misapplied particularly by the Gold Standard because they are ‘correcting’ the KPT results using the WBT results as ‘golden and the KPT results as suspect! It could not possibly be worse in terms of reputational risk. The very core of the agreement to trade money for services is founded on the WBT, without checking to see if it fit for purpose. It is an astonishing situation.

>>4.  "To make a claim such as a specific health impact from an 'amount of emissions' requires establishing the metric: emission rate? Maximum emission rate? Exposure to those rates? Average 24 hr exposure to some 'rate'? You can't leave the details to someone else. When making a claim as specific as 'this tier stove produces this health response' you need a huge amount of information that does not yet exist, and even if it did, you cannot 'calculate' an aDALY using it."

>Remember my caution "There is no there there." it is gross error - and unfair to Kirk Smith, BAMG - to assume that the HAP "Burden of Disease" is based on any data on emission rates, emission profiles on representative samples worldwide. All that was used for DALYs is the assumption of equitoxicity of PM2.5, neglect of PM2.5 other than those from household cooking fuels, and the Integrated Exposure Response curve. (Cite: Smith's Millions Dead paper and its Supplement. I can provide my in-text comments to those who request me independently.)
I agree with you, but these two paragraphs don’t cross paths.
>Much as it may aggrieve you, HAP DALYs have nothing to do with emission rates or exposures for the Millions Dead, whose lifelong exposure and disease profiles are unknown. No dose, no response, at least not in terms of actual measurements.

Understood. I was trying to communicate that aDALYs are not computable. Full stop. That you have expressed as ‘there is no ‘there’ there.

Have you heard anybody make a specific claim that 'this tier stove produces this health response' ? That is braggadocio, conceit and deceit. In fact, the very reason Gates/NIH, etc. are spending hundreds of millions of dollars on researching the impact of LPG stoves or Philips stoves is precisely because huge amounts of data are required to make such a link and even then in small populations, confounding factors make a mockery of research.
Well, yes in a clever manner. Kirk Smith has said many times, (I will cite the Bangkok conference for the GACC’s roadmap as one) that we need to reduce emissions by 90% or more in order to have a meaningful health impact. He uses this 90% mantra to support the notion that a reduction of 50% does not confer a 50% reduction in effect. It has to be 90% or more to get significant and a meaningful health gain.
The attendant claim for this is that given the level of health harm produced by a traditional solid fuel stove, doubling the emissions will have nearly no negative effect because of the aforementioned non-linear response to exposure. Unless Kirk is having it both ways: double emissions, double the harm, half the emissions, only a small reduction in harm. Is that how it works?
>There is no quantified evidence on "health benefits" of "clean cookstoves" in the West over the last 100 years.
That is believable, of it would be cited frequently.
>Similarly, there is no quantified evidence that PM2.5 concentration reductions in US over the last 40-70 years (first by fuel bans in certain locations, then technological controls) have any specific reduction in premature mortality. There are some correlations, but troubled by confounding factors.
Of that I am sure. The targets were created ‘as placeholders’ awaiting the evidence which has been rather a long time coming.
The rest of the message goes onto a separate topic so I will reply separately.
Regards
Crispin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.bioenergylists.org/pipermail/stoves_lists.bioenergylists.org/attachments/20170902/b322e193/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 20335 bytes
Desc: image001.jpg
URL: <http://lists.bioenergylists.org/pipermail/stoves_lists.bioenergylists.org/attachments/20170902/b322e193/attachment.jpg>