[Stoves] Advocacy action: ask the GACC to stop promoting the WBT

[Ronal W. Larson]

Ms. Fulland and ccs.  

	Thanks for a very informative and considerate response.

	See a few comments below.


> On Jan 27, 2017, at 7:21 AM, Camilla Fulland <camilla.fulland at primestoves.com> wrote:
> 
> Dear all, 
> 
> Thank you for all your responses and questions. 
> 
> The reason why it has taken me some time to reply is not out of lack of
> interest nor answers, but mainly due to Prime being a commercial company,
> meaning I spend my working hours selling stoves and supporting our
> distributors to make sure that our stoves are in use in the field. Rather
> than spam your inbox with several responses I have tried to summarize all
> responses here, although it has started to resemble one of Nikhil’s posts ;)
> Forgive me. 
	[RWL1:  No need to forgive;  you have provided a great complete response.
> 
> I did not write my previous post to blame any of the test centers, nor test
> center staff. They do their best within the limits of the test methods and
> their available resources. Nor did I suggest that any cookstove
> manufacturers are indeed gaming the testing, but wanted to highlight the
> possibility of this happening under the current IWA tier system and to
> provide a user perspective of testing protocols and test procedures. I am
> not a stove scientist, which is why I believe my experience reflects that of
> the majority of people involved in the cookstoves sector, from donors to
> distributors.
> 
> To Prime, success is if our customers are willing to use their hard-earned
> cash to purchase our stove, which they wouldn’t if they didn’t see the
> benefit of it, and then report back to us how it has positively impacted
> their lives by saving them money and/or time, and giving them the joy of a
> cleaner kitchen and better health. 
> 
> Our main problem with the WBT, or should I specify the IWA tiers, arises
> when this classification is used as a basis for certifying stoves under
> national standards, distributing donor money, or as a selection criterion
> for tenders etc. Prime has not conducted WBTs at various test centers to
> exploit the loopholes but because our stoves have had to be certified in
> many different countries that rely on the WBT to do so to be sold in that
> country. 
	[RWL2:  Glad you are making a distinction between the WBT and its use in placing stoves into tiers.  These are different topics.  Art Donnelly made a late Sunday AM pitch for the importance of the test protocol - which has been using to make continued improvements.  Kirk Harris has also done the same - and has achieved outstanding results - especially in obtaining a large turn-down ratio.
> 
> The market for clean cookstoves is still underdeveloped and in many cases
> some extra stimulation in the form of incentives is needed to get the market
> going. One way of structuring such an incentives system is Results Based
> Financing (RBF). Prime participates in such programs in both Indonesia (the
> CSI Indonesia by the World Bank) and the Stove Auction Cambodia by SNV. Both
> have conducted their own testing, the CSI protocol being used in Indonesia,
> and a CCT for the Stove Auction. Prime recently also qualified for the RBF
> program in Kenya, where the center for national standards tested the stove
> using the WBT. As only a tier 2 is needed to join the program, we have no
> problem to participate based on this as we have never had a test saying that
> the stove in question is less than a Tier 2. 
	[RWL3:  I will check on how the CSI, CCT and RBF relate to WBT 4.2.3 tiers.  Offhand this sounds like a cause for problems all by itself.  Anyone able to help with this comparison/relationship?  Certainly some tests give no credit for char production - by design - even for stoves trying to produce char - as Prime is (commendably) doing.  The main reason I am in this dialog. 
> 
> The problem of gaming due to the variability of test results from one test
> facility to another arises when large amounts of funds are involved and test
> and qualification criteria are very specific, like in the recent UNHCR call.
> The variability of the test results of the same stove from different test
> facilities then becomes a large problem. Until test results using the WBT,
> either from changing the protocol, the procedure of how stoves are tested,
> or the parameters that are reported, can be replicated at various test
> centers with at least 90% certainty of a tier placement, we believe that it
> is premature to 1) Allow for so many test centers to be approved, 2) Have
> such stringent tier demands in a call or a tender process, as this can
> stimulate gaming at least indirectly. 
	[RWL4:  I believe (as an outsider motivated by char properties) that your statements are in accord with the thinking of the main actors in this testing and protocol business.  Anyone disagree?
> 
> I have now gone through most of our test data. This is time consuming, as
> the testing has happened over three years and several of the spreadsheets
> have inadequate naming/description of the stove model and fuels tested. From
> this I deduct that we have at least three reasonably comparable results
> where the same model, with only superficial changes, and the same fuel was
> used. This goes for both our stove models, cylindrical and square. 
	 
> 
> To give some examples of the variability and “opposite” results, one of our
> stove models has achieved thermal efficiency scores from 30% to 56%. For low
> power PM2.5 emissions, another stove model has in one test been given a Tier
> 4, whilst in another test been given a Tier 2, both tests conducted using
> wood pellets. We do understand that the difference in fuels will affect the
> result, but we cannot support a system that cannot safely place a stove in a
> specific tier, particularly not if funds are to be paid out on this basis. 
	[RWL5:  I return to the concern that not only are you getting different results, but the tests themselves may not be the same.  Different results may be guaranteed (but I or others need to see the raw data you are looking at).  I think “others” here should mean GACC mainly.  Your difference between 30 and 56% could be due entirely to how charcoal was handled in the tier placement.  I sure hope not.
> 
> And we know we are not the only ones. I have recently heard of a Tier 3
> emissions stove being tested at Tier 0 for exactly the same stove, and I
> have also seen the test results of a now Tier 4 efficiency stove, previously
> only achieving Tier 2 (As I only saw the outside of the stove, this could be
> due to technical improvements but it seemed unlikely as the tests were
> conducted very close in time), in addition to the other manufacturers that
> have been vocal here.
	[RWL6:  Certainly we can learn a lot more with more different stoves and companies. I urge others with Ms. Fullands concern to also get your concerns (but with specifics, not generalities) into the GACC system

> As for the variability between different tests
> conducted by the same test center, I will leave that up to the experts that
> looks at our data to look closer into.
	[RWL7:  But this “same center” topic should be a key part of the discovery process.  It was a key part of the ETHOS discussions just finished (calling for 5 rather than 3 replications as a minimum).

> My point is, that until we can safely
> and with certainty place a stove in a tier, the tier system should be
> abandoned.
	[RWL8:  First lets follow your plentiful leads (and hope for others)

> It could potentially be replaced by a minimum requirement
> standard until we have a new and reliable consensus rolled out.
	[RWL9:  Hmm.  Maybe I can agree, but this is up to funders, not you or I.  GACC will be able to have a major influence here.

> If people
> like myself, although not a technical stove expert but I do work with
> cookstoves on a daily basis, find the test results impossible to comprehend,
> how do you think donors, NGOs and the general public can make rational
> decisions regarding stove performance?
	[RWL10:  You’ve well stated the problem.  I’m guessing that the solution is to use a standard test - not one for each country.  And test centers need time to come up to speed.  My perception is that this is what GACC and ISO 285 are about.  All ISO processes take a tremendous amount of time - but your data can speed that up - not slow it down.
> 
> Prime has no problem sharing all our test results and the underlying data,
> which in many cases has proved very difficult for us as a test center
> customer to obtain. We prefer to share this data on a confidential basis to
> a limited group both for and against the WBT out of respect for the test
> centers and the people working there. with 
	[RWL11:  You are now in total control.  I’d start as near the top of the GACC chain as you can - save for CEO Ms. Muhthiah.  I could be wrong but I am guessing the right person is Dr. Ranyee Chiang, who I have added as a cc.   I haven’t asked, but I am guessing this is what she would wish, after reading your note (which she probably already has).

> As I said before, we are not
> interested in blaming anyone, we are solely seeking to make some sense of
> all of this. If it can help move the sector in the right direction we may
> also allow for the conclusion of the analysis conducted by the experts
> (which certainly is not me) to be presented at ETHOS. 
	[RWL12.  Too late.  ETHOS 2018 is a year away.  I add the ETHOS chair (Elisa Derby of Winrock), who asked for exactly this sort of suggestion in her concluding remarks on Sunday.  (and I’ll try to get to some more of that soon.)    I was impressed by the professionalism on this topic shown by Ranyee and many others and think you would have been also.
> 
> What we ask for, as a user of the WBT and test facilities, is that the
> results, at least the tiers that our stove is placed in, should be the same
> regardless of which GACC accredited test facility we choose to use.
> Moreover, we insist that the test parameter “Thermal efficiency” is clearer
> defined, as investors, donors and non-stovers alike believe that thermal
> efficiency in the WBT equals trees saved, which we all know it does not.
	[RWL13:  Please explain this last “does not”.  I have spent 22 years on this stove topic because of the destruction I witnessed in Sudan from the terrible wasteful use of trees in making charcoal.  Or maybe we are in agreement on the horrible use of most charcoal for cooking.  Reports of charcoal efficiency of 20% should show that as an upper limit if produced in a device like yours - but should also show a more realistic number lie 4 or 5 % with char produced (often illegally) in the bush - with most of the profit going to the smugglers in between,
	
>  
> I really hope that the “round robin” by the GACC has decreased the
> variability between test centers and that there is full transparency of the
> process, findings and results of this, ideally presented at ETHOS and shared
> on this list. 
	[RWL14:  I (everyone) could only listen to 1/3 of the breakout presentations.  But I never heard anything about “round robins”.  Anyone?  Certainly the current GACC intent seems to do exactly that.
> 
> As for your request that Prof. Nurhuda joins the discussion, I will leave
> that up to him. Some of his engagements in Indonesia sometimes prevents him
> from taking part in public discussions like this. I trust him to respond if
> he feels he can. 
	[RWL15:  My message to him bounced.

> 
> As for myself I hope you understand that I want to help this process but
> that my main mission is to get our stoves to the users. Hence, I cannot take
> too much of an active role in this discussion other than what I have
> promised above.
	[RWL16:  Thanks again for one of the best messages ever to this list.  (I think I have read almost every one.)

Ron

> 
> 
> Camilla
> 
> Camilla Fulland
> CEO | Prime Cookstoves
> 
> camilla.fulland at primestoves.com 
> Norway: +47 48 12 05 37
> Indonesia: +628 2147 600 141
> Skype: camilla.fulland
> www.primestoves.com 

	<snipped a message unrelated to above>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.bioenergylists.org/pipermail/stoves_lists.bioenergylists.org/attachments/20170131/1c39fc8d/attachment.html>