[Stoves] Stove testing with and beyond the WBT

[Paul Anderson]

Dear Ranyee, Crispin, and all,

Thank you both for the stimulating exchanges of messages.   I think 
(hope) we are moving along the right track.

I want to pick up on only one sub-theme, about having ACTUAL DATA SETS 
for us to use in the comparisons of testing procedures. (that is NOT 
about comparisons of the specific stoves that were tested.)

>  Ranyee wrote:
>
> >Part of the discussion should be around what is the minimal set of data that the testing centers need to 
> share to facilitate collaboration and harmonization among the 
> different centers.
>
>  Crispin responded:
>
> I can open that conversation by suggesting that if a claim is publicly 
> made the raw data (in the form of the reporting spreadsheet on which 
> the calculations were made) is also made public. It may be possible to 
> limit it to that information needed to recalculate the result based on 
> what we now know are problem areas (like the treatment of fuel 
> moisture, charcoal remaining, water volume processed, averaging 
> procedures, conceptual considerations and procedural relevance.
>
> Having that information would a) allow us to recalculate the old WBT's 
> in the Stove Inventory into a modern form, b) determine what the IWA 
> tiers would be if they were set using current understanding, c) permit 
> mathematically correct averaging of multiple tests to give and 
> 'average performance', and d) permit the validation of tools that can 
> be incorporated into an ISO Standard of stove testing methods.
As far as I know (and I do NOT specialize in this), very few sets of RAW 
data are available for examination.   I am not talking about the 
inventory of tested stoves.   We do NOT need specific stoves to be 
named.  Could be "RS23B" for some Rocket Stove.   We have my partial 
data set for the Quad 2 (emissions were not measured) and it might be 
"TS32J" for TLUD Stove with a code number.   Let's have some more data sets.

1.  If I am wrong, let's have a listing of the addresses for the data.

2.  Can the EPA data from Jetter be released with code names only?    Do 
testing centers ONLY test for clients who have paid and therefore own 
their data?   Do all of the clients decline to let the testing centers 
put a few examples of full sets of raw data up for examination?   
Although some For-Profit companies might not want to share (or need 
shareholder approval?), what about the data for the stoves of all of the 
NGOs and Not-For-Profit entities that have their stoves tested?   Or 
what about raw data that was collected with funding from public money?   
Maybe the Freedom of Information Act can be applied to data files for 
stoves tested in America.

3.  Or should we just pretend that we can have great confidence in the 
existing testing?   That will lead to chaos eventually.

4.  We either have test results that will bear re-evaluation, or we do 
not.   We do not need thousands of data sets.   But a representative 
number for the major types of stoves would be appreciated.   This should 
be a PRIORITY of the GACC that makes a major statement of being 
technology neutral.

I sympathize with Ranyee who is in the "hot seat" about this and is the 
spokesperson of the GACC about testing matters.  But the GACC top 
administration needs to address this issue of having raw data sets 
available for examination (but leaving off the names of the stoves and 
stove makers).

Ranyee wrote:
>
> >Paul also raised the issue of recalculating older test results so they can be comparable or harmonized.  I think 
> another issue to consider is how much have technologies changed and 
> are the older test results still applicable.
>

Well, having "older test results" possibly deemed NON-applicable would 
really shake up the stove evaluation situation.   How old is "old"?   
Are the EPA Jetter-run tests already "old?"   We had better look into 
this NOW (action to start in April) than to have the testing issues 
haunt us for months and years and millions and millions of dollars spent 
for "technology neutral" assistance that does not back the right horses 
in the races.

That is frightening!!!!!!!!!

Paul

Paul S. Anderson, PhD  aka "Dr TLUD"
Email:  psanders at ilstu.edu   Skype: paultlud  Phone: +1-309-452-7072
Website:  www.drtlud.com

On 4/5/2013 9:09 PM, Crispin Pemberton-Pigott wrote:
>
> Dear Ranyee
>
> Thanks for joining the conversation on the Stoves list. We seem to get 
> more response than on the other site which I understand is under 
> revision. I appreciate your taking the time to read the relevant messages.
>
> >We're looking forward to seeing the detailed proposed protocol.
>
> The Tool Testing Toolbox is an approach to providing an agreed science 
> platform. We could have an ISO standard that says if you want to 
> measure thermal efficiency between the fire and the pot here is how to 
> determine it... If that method is used, the result is valid, no matter 
> what the pot size, the fuel used and the power level at the time.  It 
> is an investigative tool. As all these factors can be normalized the 
> results are directly comparable. It is however necessary to report the 
> results in a way that allows a skilled reader to understand what the 
> result claims. At the moment readers are rarely aware of what the 
> 'efficiency' number really contains. Most readers think it refers to 
> the amount of raw fuel that will be consumed performing the task 
> described. The UNFCCC has this concept written into the GEF and CDM 
> methodology. But a WBT efficiency does state the relative fuel 
> consumption. In fact the raw fuel used is not reported. It reports the 
> energy absorbed by the pot based (only) on water metrics divided by a 
> mass of dry fuel that has the same energy content as the energy 
> calculated to have been generated by the fire. This is a very 
> different figure from the dry fuel consumption, and different from the 
> raw fuel consumption.
>
> As the purpose of many projects is to 'save raw fuel' the confusion 
> results in people not getting what they are paying for (program pays, 
> fuel is not saved).
>
> >For all protocol development, we at the Alliance have been emphasizing the need for review by broad 
> stakeholders -- regional testing centers, manufacturers, investors, 
> and consumer representatives as much as possible, so we hope that the 
> review process can continue with this effort.  At one point Crispin 
> had questioned whether this process leads to independent review.
>
> It is not really dependent on my opinion. I work for several large 
> clients who want to know if they are getting what they are paying for, 
> basically. I am not allowed to provide that answer. It is evaluated 
> independently.
>
> >I think  independence is a tricky concept, because it's unclear who the independence would be from.
>
> Because it is to be independent, large clients can engage protocol 
> review organisations like TUV Rhineland, protocol development 
> assisting organisations like the Desert Research Unit or scientific 
> defenders of the community like SGS which rates performance for most 
> stove introduced into the EU. None of these organisations are involved 
> in the GACC protocol development process. They are professional 
> scientific organisations. They have experience at experiment design 
> and protocol review methods.
>
> >But I think a better way to achieve the broad goal is to have inclusive and public review (if you have enough 
> voices, the result is somewhat independent from everyone).
>
> In the past the internal review of test methods has not produced 
> reliable results. Because of this, all tests conducted in the past 
> require corrections 'forward' to be able to know what the actual test 
> result was. I have had to develop methods for doing this because 
> people have existing tests and they want to know 'what the real answer 
> is'. Money is changing hands based on the answer so it is an important 
> calculation.
>
> Some test results can be updated easily if the raw data is available. 
> If a spreadsheet with the raw data is available the correction is 
> pretty straightforward. It cannot provide all necessary metrics 
> because the WBT experiment is not constructed to provide them, but 
> major corrections can be made 'forward'. What does not work is to try 
> to introduce the calculation errors into current valid tests as a way 
> of generating a 'backward' comparison. The reason is that old tests 
> bias the result unequally not only between stoves but between fuels 
> and fuel moisture. It is very likely that such a recalculation 
> provides misleading comparisons. This means that taking a 4.2.1 test 
> result and, based on the raw data, recalculating a 4.1.2 or 3.1 result 
> gives a different output. The emissions and performance will be 
> 'significantly' different -- i.e. statistically detectable.
>
> >I think the issues to especially focus on will be ones that impact the various stakeholders - testing 
> feasibility, resources needed (time, financial, equipment), clarity 
> for communicating results, etc.
>
> I think I have addressed these issues above.
>
> >As Dean mentioned, we are focused on how to integrate field considerations into best practices for testing and 
> standards.
>
> That is good. When the discussion is broadened we can assess the 
> proposals.
>
> >We had much discussion at the Forum (notes are currently being polished), and it will continue.  It will be 
> important to work out how we fit the different protocols together, 
> when which protocol is used, based on context, resources, goals.
>
> What I am proposing is that we agree first on the concepts needed to 
> generate a relevant and reasonably precise and intercomparable test 
> result. After we have to concepts agreed, we can agree on what the 
> relevant metrics are. After that: the definitions of the metrics and 
> in which contexts they are valid. The application of measurement 
> methods requires that certain things be measured and reported. For 
> thermal efficiency, from a fuel supply point of fuel, we may have 
> 'fuel efficiency'. From a fire-to-pot point of view, we may have 
> 'energy efficiency'. From an engineering perspective we may have 'heat 
> transfer efficiency'.
>
> For each of these there are normal engineering methods which have been 
> widely agreed. We may need to adapt the terminology a bit but the 
> underlying principles are the same.
>
> One we have a number of 'tools' (the package of understandings and 
> definitions for one metric) we can conduct a valid test in segments. 
> Segments can be identified either by experience or rigorous analysis. 
> It is not very difficult. A standard or common garden variety WBT has 
> efficiency, energy consumption and power measurements, for example. 
> Simmering is a low power task at a power level suited to the stove and 
> the pot. They are separated because one cannot get the same metrics 
> from simmering as one can get from boiling (because of the lack of a 
> change in enthalpy -- heat content -- of a 'perfect simmer'). Thus two 
> tools are required each with its own definitions. Ant WBT performed 
> using validated tools would provide a valid result -- for any 
> stove-pot-fuel combination and can be reported as such.
>
> Because stove programmes are becoming more and more dependent on field 
> observations (actual performance) we can expect more pressure to 
> develop valid tools that will work in a wide variety of circumstances. 
> Indeed we are under that pressure right now. The reason is people are 
> tired of funding failed stove programmes. They fail because a) people 
> don't want to use the products or b) the product does last long enough 
> to be worth buying or c) the product does not perform as claimed or d) 
> the product is not flexible enough to cook the foods and perform the 
> tasks needed.
>
> If the test is not culturally relevant it will not provide meaningful 
> comparisons. The idea that hundreds of protocols will be needed is 
> not, however, correct. Scientific tools are already provided for 
> normalising results.
>
> >How would newly developed procedures supplement, partially replace, replace existing protocols? Of course, 
> this is also related to the goal for harmonization as raised in the IWA.
>
> Whatever process is used, the methods used in the past have not 
> provided useable tools. That is why I am awake long into the night 
> corresponding with other practitioners trying to rush to market tests 
> that are valid for the circumstances were we are working. We are 
> frequently asked for 'WBT' results and 'the real answers'. That is an 
> indictment, not of our motivations, but of our achievements to date. 
> It is a bit embarrassing!
>
> >Paul also raised the issue of recalculating older test results so they can be comparable or harmonized.  I think 
> another issue to consider is how much have technologies changed and 
> are the older test results still applicable.
>
> My concern is that the Stove Inventory is a chart of performance 
> measured with tools that provide dramatically different results for 
> the same stove. I am not exaggerating. If the use of the output number 
> is to trade Carbon offsets (and it is) then reporting that a 19% fuel 
> efficient stove is 54% efficient constitutes malpractice. That is the 
> view of the people paying the piper.
>
> >We are also planning discussions to map out a strategy for integrating future testing results and data together, 
> especially to communicate stove performance (IWA Tiers) that has been 
> independently evaluated.
>
> That is strongly seconded by me!
>
> >Part of the discussion should be around what is the minimal set of data that the testing centers need to 
> share to facilitate collaboration and harmonization among the 
> different centers.
>
> I can open that conversation by suggesting that if a claim is publicly 
> made the raw data (in the form of the reporting spreadsheet on which 
> the calculations were made) is also made public. It may be possible to 
> limit it to that information needed to recalculate the result based on 
> what we now know are problem areas (like the treatment of fuel 
> moisture, charcoal remaining, water volume processed, averaging 
> procedures, conceptual considerations and procedural relevance.
>
> Having that information would a) allow us to recalculate the old WBT's 
> in the Stove Inventory into a modern form, b) determine what the IWA 
> tiers would be if they were set using current understanding, c) permit 
> mathematically correct averaging of multiple tests to give and 
> 'average performance', and d) permit the validation of tools that can 
> be incorporated into an ISO Standard of stove testing methods.
>
> Best regards
>
> Crispin
>
>
>
> _______________________________________________
> Stoves mailing list
>
> to Send a Message to the list, use the email address
> stoves at lists.bioenergylists.org
>
> to UNSUBSCRIBE or Change your List Settings use the web page
> http://lists.bioenergylists.org/mailman/listinfo/stoves_lists.bioenergylists.org
>
> for more Biomass Cooking Stoves,  News and Information see our web site:
> http://stoves.bioenergylists.org/
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.bioenergylists.org/pipermail/stoves_lists.bioenergylists.org/attachments/20130406/aab4de45/attachment.html>