[Stoves] Back into action

Fri Sep 8 10:19:23 CDT 2017

Dear Tom,

I don’t have the answer to your questions. I don’t know.

See, how easy it is to admit one’s own ignorance? I wish « experts » would do that.

Just like neither Ron, nor you, nor Dean Still, nor me, nor anyone I ever questioned about the WBT issues are able answer them, to find solutions to solve them.

I didn’t phrase my sentence correctly in my previous post, I said:

“There was no success with the WBT in making meaningful and lasting social and environmental impacts (see the various articles from the press about the failure of the cookstove sector as a whole).”

I should have said: « I assume that a big part of the failure of the cookstove sector is to be attributed to difficulties to test and develop models that respond to a local need (models that sell in large numbers). For the most part, because the WBT is unreliable and we relied too much on it. So I consider that the WBT has failed in making meaningful and lasting social and environmal impacts (see the various articles from the press about the failure of the cookstove sector as whole). »

I don’t know how much harm the WBT did, how much better alternative methods would allow to make a social and environmental impact.

I did answer to Nikhil this in my post of the 24/06/2017:

« « Has WBT use harmed anybody? I am still waiting for an answer. »

As you can imagine, this is difficult to answer. The negative impact has never been measured, and it is hard to measure. Because it is about lost opportunities, it is opportunity cost.

I have been struggling for weeks in Benin, back in 2011, performing WBT with different designs of charcoal stoves, the ones from my workshop, as well as traditional or improved ones. CCTs helped me understand better, and eventually I did some sort of KPT, I left stoves at neighbours houses for days, which gave me much more usable data. It was the same with the institutional stoves.

At Prakti, we were also having difficulties with WBT testing, back in 2013 and when working on a charcoal model. We started to focus more on wood and multi fuel stoves and Jiddu and the research team increasingly implemented the HTP in the lab. It helped a lot. 

Vahid and Camilla mentioned how the WBT tests made their work at ILF and Prime Stove difficult.

Crispin mentioned this USaid project in 2007-2008 which was a big failure, largely because of initial incorrect Water Boiling Tests results. Crispin thinks because of this failure, the USaid was reluctant to conduct stove projects for a while.

I am sure there are other examples. »

That’s all I’ve got.

Then some may say: « fine, we are not able to prove that the WBT is harmful to the sector, so let’s keep using it. »

I say we can test stoves by rolling the dice, it is much simpler, and much more fun. Stoves will be tested, results will be generated, stoves will be developed and launched and sold or not on the market. We’ll not be able to know if it did have a positive or negative impact on the sector, so we are fine. I’ll say let’s just use dice to test stoves.

The WBT is broken, it is bad science, it is not reliable, this has been proven.

This is all that matters, there is no way we can use it one more second knowing all that, not when there are alternatives.

Best,

Xavier

De : Stoves [mailto:stoves-bounces at lists.bioenergylists.org] De la part de tmiles at trmiles.com
Envoyé : vendredi 8 septembre 2017 16:31
À : 'Discussion of biomass cooking stoves'; 'Frank Shields'
Objet : Re: [Stoves] Back into action

What proportion of stoves are funded through programs that use current methods for selection? How many households are impacted? 

What proportion of stoves are disseminated by independent individuals and organizations who develop stoves suited to local markets and who ignore, or are independent of, the larger organizations? What are the numbers? 

What is the “cost” of the current procedures in terms of the number of stoves disseminated, the extent of the population affected, and the economic and health impact of the beneficiaries?

How would alternative methods reduce the cost and increase the benefits of improved stoves? By how much? 

Tom

From: Stoves [mailto:stoves-bounces at lists.bioenergylists.org] On Behalf Of Xavier Brandao
Sent: Friday, September 08, 2017 6:52 AM
To: Frank Shields <franke at cruzio.com>
Cc: Discussion of biomass cooking stoves <stoves at lists.bioenergylists.org>
Subject: Re: [Stoves] Back into action

Dear Frank,

By WBT program, do you mean protocol?

I still disagree. 

"The program allowed any stove maker with a few dollars to have their stove tested and listed along with all others."

"It did ‘compare stoves’ and it did allow a stove(s) to be picked from a long list for the Real World."

But what good is it if we cannot trust the results?

My Dice Roll Test (DRT) also allows any stove maker to have their stove tested and listed along with all others, it allows to compare stoves, and it is totally free.

"The lab doing the testing made money from the client such that the lab was profitable and could continue."

Then these labs were commercially successful and also successful in terms of how much audience they reached. They did consultancy work, were associated to large programmes. There is some success in that.

But they made money "selling results" that were not reliable (note that I don't say that it was done out of malice).

There was no success with the WBT in making meaningful and lasting social and environmental impacts (see the various articles from the press about the failure of the cookstove sector as a whole).

And the most important for all of us, is to change the world, not to make money on testing or consultancy contracts.

So I would say the WBT protocol failed.

"If the WBT doesn’t work lets drop the WBT."

Amen. Now we have all the proofs the WBT doesn't work.

If anyone can contradict these proofs with solid evidence that it does work, now is the time to discuss and work on it.

Best,

Xavier

On 8 September 2017 at 04:43, Frank Shields <franke at cruzio.com> wrote:

Dear Xavier, Stovers,

On Sep 7, 2017, at 12:46 PM, Xavier Brandao <xav.brandao at gmail.com> wrote:

“The WBT was a successful program to compare stoves and allow the stoves with the best potential to be picked for further consideration in the real World."

part one:

“The WBT was a successful program 

The program allowed any stove maker with a few dollars to have their stove tested and listed along with all others. 

The lab doing the testing made money from the client such that the lab was profitable and could continue. . 

I call that a successful program.

to compare stoves and allow the stoves with the best potential to be picked for further consideration in the real World.”

It did ‘compare stoves’ and it did allow a stove(s) to be picked from a long list for the Real World. 

as to “best potential”  - well that needed to be worked on. 

This is all NORMAL for such a program. Its a set up  that is working and designed such that changes can be made to improve on making the system better at determining the stove with the  ‘Best Potential’. This is ongoing forever and ever….  Thats how science works!

But this group screwed up. 

If the WBT doesn’t work lets drop the WBT. Lets drop the entire successful system in place to make improvements. Now we have no system in place at all. A new stove maker or design that might use electric controls or fans that run off changes in heat will not see the light of day. Or be very expensive to do the only thing - get some group to field test somewhere. 

It seems the study that is now being introduced is akin to studying the ‘evolution of mankind’. : )

Regards

Frank

No, it wasn't. The WBT was not successful, it was popular. Very popular. Successful and popular are not the same.

Stakeholders who used the WBT thought it allowed them to compare and pick the stoves with the best potential for the real world.

Since the metrics were invalid and the results absolutely unreliable, comparing them meant nothing.

But sure, the WBT is incredibly simple to use, and makes one think it can do something it cannot: no wonder it's been popular.

Now, I think for those who do not want to consider other protocols because the WBT is easy and practical to use, I can suggest another testing protocol.

It is actually way easier in its use than the WBT, so then it is way better.

It is called the Dice Roll Test (DRT).

Some say that the WBT is legitimate because it's been used for decades. Well the Dice Roll Test has been used by various human civilizations for thousands of years. You cannot beat such legitimacy.

The DRT is simple, you roll the dice to determine the fuel savings and emissions (CO, PM, PM 2.5) of the stove. Then you record the results on a sheet of paper. That’s it.

This allows comparability, and placing stoves into Tiers as well.

That's what makes it so great, and you can generate a lot of data in no time.

Like the WBT, the more tests you do with the DRT, the more probabilities you have that one test will represent accurately the performance. I am sure mathematicians could argue the approach of the protocol has some perfectly defendable merits.

So, based on the above compelling arguments, I recommend that the DRT replaces the WBT starting from now.

More seriously: testing protocols have to be more than just easy to use. They are testing protocols to produce and disseminate millions of stoves, for christ sakes, it has to be really serious.

It is ok if I need to be in a fully equipped lab rather than in my backyard, and if I need few days rather than one afternoon to do my testing. This is an important matter, it is normal engineers and researchers have to be mobilized for that. It is way too important.

When did easy became more important than scientifically valid?

"Lets correct these problems and get the WBT back to central to our many faceted program."

Do you think these problems can be corrected Frank?

What would be your approach, for each issue described below this email, to correct them for sure and once and for all?

Best,

Xavier

« Some of WBT critical issues remain unsolved. In particular, the main weakness of the WBT concerns its real-life relevance. As a matter of fact, the protocol prescribes to test the stove for a fixed combination of burn sequence (high-power and low-power) and pot dimensions. This limits the test relevance to just that particular setting: when any variation of these parameters e which deeply affect the system performance e is introduced, results may significantly differ. Criticism about WBT concerns also the repeatability of the protocol, with a number of researchers claiming that it would need to be reviewed in terms of accuracy. The WBT is a controlled laboratory test, thus supposed to be characterized by good repeatability and to be effective in comparing different stove designs. Nevertheless, the choice to bring the water to the boil preventing the use of the lid e that is made in order to better approximate a typical cooking task e is not functional to this purpose. As a matter of fact, uncertainties related to temperature reading and vaporisation in the boiling region lead to high variability between test replicates. In order to improve the reliability and replicability of the tests, the latest version of the protocol includes the section “Changes to Testing Conditions to Improve Repeatability”, which refers to different fuels and pot characteristics. However, eventual changes involving other parameters (pot insulation and maximum water temperature), which deeply affect test variability, are not mentioned. Such problems give evidence of an unsolved conflict between the declared purpose of the WBT (viz. to be a design-phase test, not intended to be representative of real-use performance), and a general tendency to use the results form WBT as a significant means to select the most appropriate cooking stove for a given context. A lot of debate has been made around formulation of metrics, primarily on thermal efficiency (Table C3), which is often interpreted as the most immediate and distinctive stove performance parameter. Studies from Bailis et al. highlighted how relying on WBT thermal efficiency outputs, regardless of the relative importance of high and low power cooking tasks among the target population, can lead to misleading interpretations. Furthermore, Zhang et al. and Jetter et al. questioned the scientific meaningfulness of thermal efficiency at simmering. Indeed, this phase is characterised by highly variable steam production, which represents a heat loss in the energy balance but positively contributes to the efficiency value in the actual formulation of thermal efficiency. Finally, some unsolved issues concerning statistical significance of data are worth mentioning. WBT 4.2.3 includes “Statistic Lessons for Performance Testing”. The appendix specifies that the minimum number of test replicates for each model of stove should be three, although it is reported that this number of replicates is not necessarily sufficient to determine a stove performance within a certain confidence interval. Nevertheless, Wang et al. noticed how a great majority of published studies are performed using a number of replicates that is equal or less than three, perhaps due to a misinterpretation of the Appendix message as “only three tests are needed”, regardless of variability and confidence interval. Wang et al. investigated this topic using a simplified version of the WBT 3.0 and demonstrated that more than 5 replicates are likely to be required to avoid impractically large 95% confidence intervals and that even more replicates may be required to demonstrate a statistically significant difference in performance between two or more stoves. »

-----Message d'origine-----
De : Stoves [mailto:stoves-bounces at lists.bioenergylists.org] De la part de Frank Shields
Envoyé : mercredi 6 septembre 2017 17:03
À : Discussion of biomass cooking stoves
Objet : [Stoves] Back into action

Stovers,

The WBT was a successful program to compare stoves and allow the stoves with the best potential to be picked for further consideration in the real World. It was only flawed in the test method used. Lets correct these problems and get the WBT back to central to our many faceted program.  

1) It needs to relate to the fuel planned to be used. I suggest the energy in the Volatile Fraction to be the total energy released to the secondary. The Fixed energy to be the maximum  char making capability of the fuel. Results of percent energy used (if that is at all important?) is based on the volatile energy fraction and the Percent char produced a percent of the maximum for that biomass. That will please all three groups; TLUD, Rocket and Ron : ).

2) We need to have it measured for the many different tasks that the stove might be used. I suggest developing a Task Cube. This a solid metal (platinum, aluminum, steel, titanium) of something that will last. Cubed to exact measurements, Drilled and taped to fit a thermistor to record the increase in temperature. Place this block in with different foods being cooked and record the temperature profile. Develop the profile that needs be obtained for each task. Sell the Task Cubes to labs where they can place them in boiling water or sit on a hot surface then follow the temperature profile to determine if the stove will work for that task and when the task is completed.  

So we go from the actual fuel (well sort of) to the actual task (well sort of). The program is (or was) already set up and used so should be quick to get it running again. Purpose it to provide Cecil Cook (and others) with a list of stoves from an unbiased source and with best chance for success into the field so they can do the things they do. And provide the opportunity for every stove designer to get their stove  compared with others.  

Thanks

Frank

Frank Shields

Gabilan Laboratory

Keith Day Company, Inc.

1091 Madison Lane <https://maps.google.com/?q=1091+Madison+LaneSalinas,+CA%C2%A0+93907+%28831&entry=gmail&source=g> 

Salinas, CA <https://maps.google.com/?q=1091+Madison+LaneSalinas,+CA%C2%A0+93907+%28831&entry=gmail&source=g>   93907 <https://maps.google.com/?q=1091+Madison+LaneSalinas,+CA%C2%A0+93907+%28831&entry=gmail&source=g> 

(831) 246-0417 <tel:%28831%29%20246-0417>  cell

(831) 771-0126 <tel:%28831%29%20771-0126>  office

 <mailto:franke at cruzio.com> franke at cruzio.com

_______________________________________________

Stoves mailing list

to Send a Message to the list, use the email address  <mailto:stoves at lists.bioenergylists.org> stoves at lists.bioenergylists.org

to UNSUBSCRIBE or Change your List Settings use the web page  <http://lists.bioenergylists.org/mailman/listinfo/stoves_lists.bioenergylists.org> http://lists.bioenergylists.org/mailman/listinfo/stoves_lists.bioenergylists.org

for more Biomass Cooking Stoves,  News and Information see our web site:

 <http://stoves.bioenergylists.org/> http://stoves.bioenergylists.org/

Thanks

Frank

Frank Shields

Gabilan Laboratory

Keith Day Company, Inc.

1091 Madison Lane <https://maps.google.com/?q=1091+Madison+LaneSalinas,+CA+%C2%A093907+%28831&entry=gmail&source=g> 

Salinas, CA  <https://maps.google.com/?q=1091+Madison+LaneSalinas,+CA+%C2%A093907+%28831&entry=gmail&source=g>  93907 <https://maps.google.com/?q=1091+Madison+LaneSalinas,+CA+%C2%A093907+%28831&entry=gmail&source=g> 

(831) 246-0417 <tel:%28831%29%20246-0417>  cell

(831) 771-0126 <tel:%28831%29%20771-0126>  office

fShields at keithdaycompany.com

franke at cruzio.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.bioenergylists.org/pipermail/stoves_lists.bioenergylists.org/attachments/20170908/752b96e1/attachment.html>