[Stoves] Advocacy action: ask the GACC to stop promoting the WBT

Mon Jan 23 12:08:14 CST 2017

Dear Ron,

*[RWL1: Could be, but changes are supposed to be much easier than the 
first approval - I hear.  Can you make a guess as to how long starting 
over would take?*
I don't know when the ISO process for cooking stoves started, but more 
than 6 years ago?
My guess estimate: twice more to change the standards once they are 
adopted, so ... 12 years?
Now, there is momentum, the members of the WG are working together. How 
quick can WG get to work together again once the standards have been 
adopted and set in stone?
How long to restart this huge machinery?

*I haven’t heard your view on that topic.**
*I am not very opinionated on that subject. I agree that the charcoal 
produced could be considered as a "benefit"**given by the stove, but 
there may be other ways to account for it. I agree this remaining 
charcoal should be accounted.*

*Even if this question is still under discussion, the WBT issues with 
the questions of boiling and simmering, the unsure repeatability, the 
results leading to potentially very large errors render it useless.*
*
*[RWL4:  I am sure there are many critiquing** WBT 4.2.3, but no-one has 
yet supplied a reference on the “denominator equation” - which I 
believe is the ONLY way to get valid stove comparisons.  I’m anxious to 
see their reason why -  Crispin has yet to give me a reason - much less 
a convincing one.  Nor anyone else.  Can you give it a try?  What would 
you replace this equation with?
*I don't know with what I would replace this equation, I am not a 
scientist. I think it should be discussed on the List, like you are 
doing now, that is good.
But whatever the results and the duration of this discussion, what 
matters is that it is not the only issue with the WBT, as mentioned 
above, and all the other issues pointed out by the studies have never 
been contradicted.**All the other issues are enough to decide to stop 
using the WBT.*
*
*I think the stove industry will generally be happy with three tests - 
especially if they are paying for them.
*On the contrary, I think the stove industry will be very unhappy with 
three tests that does not allow to know at all how their stove perform.
Especially if they are paying for them.

*I found very little to justify big changes in 4.2.3**
*Really?! It doesn't raise any alarm in your mind when authors write:

"However, different authors have been raising doubts about the 
consistency of WBT results, focusing in particular on three issues: (i) 
L’Orange et al. [6] highlighted the role of thermodynamic uncertainties 
(viz. variable steam production and boiling point determination) on 
results repeatability; (ii) Zhang et al. [7] raised questions about the 
rationale of some calculations and about metrics terminology; (iii) 
finally, Wang et al. [8] criticised the statistical approach recommended 
by this standardised laboratory-based test to evaluate, communicate and 
compare performances and emissions of tested stoves, i.e. using the 
arithmetic average of three replicate tests.

The results suggest how considering only the mean values of the outputs 
of the WBT and neglecting intrinsic uncertainties of the results may 
lead to make large errors and misinterpretations regarding the ICSs’ 
performance. Indeed, for all the three Classes analysed, at 90% degree 
of confidence, the percentage of ‘‘improved” stoves obtained by 
considering the mean values of the WBT is among 3 and 6 times higher 
than the percentage resulted from this analysis at least. At 99% 
confidence level, only 15% of all the supposed ‘‘improved” stoves 
emerged as real ICSs at most. When enough statistical information is 
provided from WBT results, only the Stove with fan model of cookstoves 
seemed to reveal real improvements with a probability greater than 93%. 
This work shows how neglecting the epistemic statistical uncertainties 
originated from WBTs – as done by a large portion of the literature, 
which reports results from few lab-tests replicates without sufficient 
statistical information – might lead to misinterpreted evaluations of 
ICSs’ performance, with potential negative impact on beneficiaries.

The results suggest how considering only the mean values of the outputs 
of the WBT and neglecting intrinsic uncertainties of the results may 
lead to make large errors and misinterpretations regarding the ICSs’ 
performance. Indeed, for all the three Classes analysed, at 90% degree 
of confidence, the percentage of ‘‘improved” stoves obtained by 
considering the mean values of the WBT is among 3 and 6 times higher 
than the percentage resulted from this analysis at least. At 99% 
confidence level, only 15% of all the supposed ‘‘improved” stoves 
emerged as real ICSs at most. When enough statistical information is 
provided from WBT results, only the Stove with fan model of cookstoves 
seemed to reveal real improvements with a probability greater than 93%. 
This work shows how neglecting the epistemic statistical uncertainties 
originated from WBTs – as done by a large portion of the literature, 
which reports results from few lab-tests replicates without sufficient 
statistical information – might lead to misinterpreted evaluations of 
ICSs’ performance, with potential negative impact on beneficiaries."

**There is something rotten in the WBT, and small changes won't change 
anything to it. Small changes and updates have happened on a regular 
basis these last years, to no avail.

I think it is good that you want to bring answers to your denominator 
equation, and won't stop until you do. That is important.
I also think that we don't need to answer that question to move 
forward**on the WBT question.*
*
Best,*

*Xavier*
*

On 1/23/17 02:54, Ronal W. Larson wrote:
> Xavier and List:
>
> See inserts below.
>
> Briefly summarizing -  none of your recommended papers gave any 
> insight into the validity of the “denominator equation”.  Before 
> anyone sends me additional papers related to any Water Boiling Test, 
>  I would greatly appreciate being told that they address this 
> charcoal-value equation in some way.
>
>
>> On Jan 19, 2017, at 5:11 PM, Xavier Brandao <xvr.brandao at gmail.com 
>> <mailto:xvr.brandao at gmail.com>> wrote:
>>
>>
>>
>> Dear Ron,
>>
>> You are raising interesting points.
>>
>> Thanks for sharing the documents of the Lima consensus. It was in 
>> 2011. Now we are in 2017. Signatories might have changed mind.
>>
>> Please note that there is also a certain state of mind with people 
>> working on the ISO:"ISO standards are good for the stove sector. 
>> Therefore we have to reach an agreement, as soon as possible. Also we 
>> put a lot of work in it, so we'd like to be done with it."
>>
>> This is understandable, but one should not take decisions in haste, 
>> without firm scientific basis. Once the ISO standards will be set, 
>> it'll be too late. There won’t be a way to change them, or it'll take 
>> years and years.
> *[RWL1: Could be, but changes are supposed to be much easier than the 
> first approval - I hear.  Can you make a guess as to how long starting 
> over would take?*
>
>> So, OK to set standards for stoves, but not at all cost. That's what 
>> Crispin has been trying to say, but he has too often be seen as a 
>> hindrance.My opinion: I'd rather have no standards than bad 
>> standards. Bad standards would be devastating. It doesn’t matter if 
>> it takes time, ISO standards have to be scientifically valid, or they 
>> should not exist.
>>
> *[RWL2:   My involvement in this process has been engendered by a 
> concern that char-making stoves (which I see as being desired for many 
> applications) can be hurt through modifying 4.2.3 re 
> this “ denominator equation”.  So far that has not happened.  But I 
> see rejection of the “denominator equation” as having that potential. 
>  I consider that equation to be “scientifically valid”.  I haven’t 
> heard your view on that topic.*
> *See more notes on the seven attachments - for which I thank you.
> *
>>
>> If they are still nowadays supporting the WBT, I’d like to hear the 
>> signatories of the consensus tell us on this list why the WBT is a 
>> good protocol, and why we should keep using it.
>>
>
> *[RWL3:  As I said earlier - I will be reporting this in about a week*
>>
>> "And, from what I hear, essentially one person alone (Crispin) is 
>> critiquing it."
>>
>> Crispin is not the only one, trust me. We are a big group, and it is 
>> growing. His critics of the WBT find an echo especially with the 
>> project implementers, who find the WBT inadequate, and are struggling 
>> with testing. Implementers are quiet on the Stove List, they are very 
>> busy with day-to-day operations. So was I during more than 3 years 
>> when I was with Prakti in India. Plus, the state of mind of 
>> implementers is: "anyways, Crispin is on top of that, he is leading 
>> the fight, and people in the TC 285 are probably discussing these 
>> matters. This is the job of researchers, this is not our job.”
>>
>
> *[RWL4:  I am sure there are many critiquing** WBT 4.2.3, but no-one 
> has yet supplied a reference on the “denominator equation” - which I 
> believe is the ONLY way to get valid stove comparisons.  I’m anxious 
> to see their reason why -  Crispin has yet to give me a reason - much 
> less a convincing one.  Nor anyone else.  Can you give it a try?  What 
> would you replace this equation with?*
>
>> Please find a list of the studies where the issues with the WBT have 
>> been mentioned or studied in detail:
>>
>> //
>> ·/Fuzzy interval propagation of uncertainties in experimental 
>> analysis for improved and traditional three–stone fire 
>> cookstoves,/Riva F., Lombardi F., Pavarini C., Colombo E., 07/09/2016
> *[RWL5:   I have not gone into this one in detail - as there is no 
> mention of the “denominator equation”.  My concern is that I don’t see 
> a discussion of the costs of achieving very high reliability on a 
> result.  I think the stove industry will generally be happy with three 
> tests - especially if they are paying for them.*
> *Also, the emphasis here is only on the efficiency computations - 
> whereas the hard (expensive) part of all the WBT measurements is the 
> data on CO and particles.
> *
>> ·/Key differences of performance test protocols for household biomass 
>> cookstoves,/Twenty-Second Domestic Use of Energy, IEEE 2014:1–11, 
>> Zhang Y, Pemberton-Pigott C, Zhang Z, Ding H, Zhou Y, Dong R., 2014
> *[RWL6:  Similar - also trying admirably for high precision - but I 
> think asking for more than is needed.  Again only looking at 
> efficiencies - and never on the “denominator equation.*
>
>> //
>> ·/Performance testing for monitoring improved biomass stove 
>> interventions: experiences of the Household Energy and Health 
>> Project/. Energy SustainableDev 2007; 11:57–70. Bails R, Berrueta V, 
>> Chengappa C, Dutta K, Edwards R, Masera O, et al., 2007,
> *[RWL7:  This had a fee, but I am part of a group able to get some 
> papers like this.  I found nothing on the “denominator equation”.*
>
>> //
>> ·/The shortcomings of the U.S. protocol/, Robert Pendleton Taylor, 2009
> *[RWL8:   This definitely looking at the “denominator equation” (that 
> I am trying to protect) - but NOT finding fault with it on theoretical 
> grounds.  Only writing on the big inaccuracies possible with if one 
> assumes that the energy density of char is always 1.5 times that for 
> biomass.   I agree - and this seems to be supportive - as this does 
> not today apply to 4.2.3.  Both energy densities are today being 
> measured (at considerable extra expense) - never assumed.*
>> ·/Influence of testing parameters on biomass stove performance and 
>> development of an improved testing protocol/. Energy Sustainable Dev 
>> 2012; 16:3–12. L’Orange C, DeFoort M, Willson B., 2012
> *[RWL9:   Not sent, but downloadable (non-fee) at 
> *https://envirofit.org/wp-content/uploads/2016/06/2012-influence-of-testing-parameters.pdf
>
> I enjoyed this paper;  much good background.  Here is a pertinent 
> sentence: /An energy transfer test such as the WBT or the revised 
> protocol presented here, alternatively, allows for more control and 
> repeatability but may fail to capture the nature of how the stove will 
> perform during real use/
>
> Unfortunately, this paper also does not mention the “denominator 
> equation”.
>
>> ·/How many replicate tests are needed to test cookstove performance 
>> and emissions? — Three is not always adequate/. Energy Sustainable 
>> Dev 2014; 20:21–9. Wang Y, Sohn MD., 2014
> *[RWL10:   This one not sent to me.  I found (non-fee) at 
> *http://escholarship.org/uc/item/7nz614g4 .  The full list of authors is
> Yungang Wang1,, Michael D. Sohn1 , Yilun Wang2 , Kathleen M. Lask3 , 
> Thomas W. Kirchstetter1,4, Ashok J. Gadgil1,
>
> Here is a key sentence on the number issue: /A literature survey of 
> recent laboratory cookstove testing in peer-reviewed journal articles 
> shows widely different numbers of replicate tests (Bailis et al., 
> 2007; Jetter and Kariher, 2009; Jetter et al., 2012; McCarty et al., 
> 2008, 2010; Roden et al., 2009; Smith et al., 2007)/.
>
> As I understand 4.2.3, the number “3” is not mandated.  I can well 
> understand wanting extreme precision in every data point - but this 
> can be achieved by looking mostly at the top potential winning stoves 
> if there is a competition.  I see little reason for this precision for 
> stoves that need improvement.  There is a strong preference here for 5 
> replications.
>
> The “denominator equation” does not appear at all.
>
>> ·/Systematic and conceptual errors in standards and protocols for 
>> thermal performance of biomass stoves/, Zhang Z, Zhang Y, Zhou Y, 
>> Riaz A., Pemberton-Pigott C, Annegarn H., Dong R., 2014
> *[RWL11:   This expressed the greatest concerns - mostly allied with 
> yours.  But again, only (I think) two oblique mentions of 
> the “denominator equation”,  using the word “charcoal” only twice - in 
> neither case relative to this equation.*
> *I also see mostly (entirely?) here a concern on what is reported - 
> NOT on what is measured nor how it is measured.*
> T*wo others of the seven you sent are subsets of this one.*
>> ·/Quality assurance for cookstoves testing centers: calculation of 
>> expanded uncertainty for WBT/, Gorily M., Trujillo G., 2013
> *[RWL12:   Not received - and I couldn’t find by googling.  I hope 
> someone can send me this one.*
>>
>> I sent you some papers by email.I believe there are other sources as 
>> well, I'll keep gathering more data.
>>
>> To my knowledge, none of these papers has been discussed and 
>> criticized on a scientific basis by WBT supporters. I am assuming it 
>> is because they make a point.
> *[RWL13:  Well now I have critiqued one set.   I found very little to 
> justify big changes in 4.2.3 - and nothing related to my reason for 
> wanting to defend the “denominator equation”.     The next sentence is 
> mine and I still stand by it.  I believe all of the papers you have 
> cited are making relatively small corrections to 4.2.3 - such as the 
> number of replications and suggesting small changes to outputs*
>>
>> "I contend that lab testing is imperative. I contend that there is no 
>> better lab test that boiling water.Who has a better approach?"
>>
>> There are other protocols for lab testing: at Prakti, we have been 
>> using a slightly modified version of the Heterogenous Testing 
>> Protocol. We use contextual testing as well. I believe the 
>> Uncontrolled Cooking Test and the CSI Water Heating Test are good as 
>> well. They do everything a WBT does.
>>
> *[RWL14:  Please send me a copy of these three or cites - but only if 
> they have an alternative to the “denominator equation”.  None of these 
> three employ water boiling?*
>>
>> Best,
>>
>> Xavier
>>
>>
>> _______________________________________________
>> Stoves mailing list
>>
>> to Send a Message to the list, use the email address
>> stoves at lists.bioenergylists.org <mailto:stoves at lists.bioenergylists.org>
>>
>> to UNSUBSCRIBE or Change your List Settings use the web page
>> http://lists.bioenergylists.org/mailman/listinfo/stoves_lists.bioenergylists.org
>>
>> for more Biomass Cooking Stoves,  News and Information see our web site:
>> http://stoves.bioenergylists.org/
>>
>
>
>
> _______________________________________________
> Stoves mailing list
>
> to Send a Message to the list, use the email address
> stoves at lists.bioenergylists.org
>
> to UNSUBSCRIBE or Change your List Settings use the web page
> http://lists.bioenergylists.org/mailman/listinfo/stoves_lists.bioenergylists.org
>
> for more Biomass Cooking Stoves,  News and Information see our web site:
> http://stoves.bioenergylists.org/
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.bioenergylists.org/pipermail/stoves_lists.bioenergylists.org/attachments/20170123/a0017f19/attachment.html>