[Stoves] Stove testing with and beyond the WBT

Fri Apr 5 21:09:11 CDT 2013

Dear Ranyee

Thanks for joining the conversation on the Stoves list. We seem to get more
response than on the other site which I understand is under revision. I
appreciate your taking the time to read the relevant messages.

>We're looking forward to seeing the detailed proposed protocol.  

The Tool Testing Toolbox is an approach to providing an agreed science
platform. We could have an ISO standard that says if you want to measure
thermal efficiency between the fire and the pot here is how to determine it.
If that method is used, the result is valid, no matter what the pot size,
the fuel used and the power level at the time.  It is an investigative tool.
As all these factors can be normalized the results are directly comparable.
It is however necessary to report the results in a way that allows a skilled
reader to understand what the result claims. At the moment readers are
rarely aware of what the 'efficiency' number really contains. Most readers
think it refers to the amount of raw fuel that will be consumed performing
the task described. The UNFCCC has this concept written into the GEF and CDM
methodology. But a WBT efficiency does state the relative fuel consumption.
In fact the raw fuel used is not reported. It reports the energy absorbed by
the pot based (only) on water metrics divided by a mass of dry fuel that has
the same energy content as the energy calculated to have been generated by
the fire. This is a very different figure from the dry fuel consumption, and
different from the raw fuel consumption. 

As the purpose of many projects is to 'save raw fuel' the confusion results
in people not getting what they are paying for (program pays, fuel is not
saved).

>For all protocol development, we at the Alliance have been emphasizing the
need for review by broad stakeholders - regional testing centers,
manufacturers, investors, and consumer representatives as much as possible,
so we hope that the review process can continue with this effort.  At one
point Crispin had questioned whether this process leads to independent
review.  

It is not really dependent on my opinion. I work for several large clients
who want to know if they are getting what they are paying for, basically. I
am not allowed to provide that answer. It is evaluated independently.

>I think  independence is a tricky concept, because it's unclear who the
independence would be from.  

Because it is to be independent, large clients can engage protocol review
organisations like TUV Rhineland, protocol development assisting
organisations like the Desert Research Unit or scientific defenders of the
community like SGS which rates performance for most stove introduced into
the EU. None of these organisations are involved in the GACC protocol
development process. They are professional scientific organisations. They
have experience at experiment design and protocol review methods.

>But I think a better way to achieve the broad goal is to have inclusive and
public review (if you have enough voices, the result is somewhat independent
from everyone).  

In the past the internal review of test methods has not produced reliable
results. Because of this, all tests conducted in the past require
corrections 'forward' to be able to know what the actual test result was. I
have had to develop methods for doing this because people have existing
tests and they want to know 'what the real answer is'. Money is changing
hands based on the answer so it is an important calculation.

Some test results can be updated easily if the raw data is available. If a
spreadsheet with the raw data is available the correction is pretty
straightforward. It cannot provide all necessary metrics because the WBT
experiment is not constructed to provide them, but major corrections can be
made 'forward'. What does not work is to try to introduce the calculation
errors into current valid tests as a way of generating a 'backward'
comparison. The reason is that old tests bias the result unequally not only
between stoves but between fuels and fuel moisture. It is very likely that
such a recalculation provides misleading comparisons. This means that taking
a 4.2.1 test result and, based on the raw data, recalculating a 4.1.2 or 3.1
result gives a different output. The emissions and performance will be
'significantly' different - i.e. statistically detectable.

>I think the issues to especially focus on will be ones that impact the
various stakeholders - testing feasibility, resources needed (time,
financial, equipment), clarity for communicating results, etc.

I think I have addressed these issues above. 

>As Dean mentioned, we are focused on how to integrate field considerations
into best practices for testing and standards.  

That is good. When the discussion is broadened we can assess the proposals. 

>We had much discussion at the Forum (notes are currently being polished),
and it will continue.  It will be important to work out how we fit the
different protocols together, when which protocol is used, based on context,
resources, goals.  

What I am proposing is that we agree first on the concepts needed to
generate a relevant and reasonably precise and intercomparable test result.
After we have to concepts agreed, we can agree on what the relevant metrics
are. After that: the definitions of the metrics and in which contexts they
are valid. The application of measurement methods requires that certain
things be measured and reported. For thermal efficiency, from a fuel supply
point of fuel, we may have 'fuel efficiency'. From a fire-to-pot point of
view, we may have 'energy efficiency'. From an engineering perspective we
may have 'heat transfer efficiency'. 

For each of these there are normal engineering methods which have been
widely agreed. We may need to adapt the terminology a bit but the underlying
principles are the same.

One we have a number of 'tools' (the package of understandings and
definitions for one metric) we can conduct a valid test in segments.
Segments can be identified either by experience or rigorous analysis. It is
not very difficult. A standard or common garden variety WBT has efficiency,
energy consumption and power measurements, for example. Simmering is a low
power task at a power level suited to the stove and the pot. They are
separated because one cannot get the same metrics from simmering as one can
get from boiling (because of the lack of a change in enthalpy - heat content
- of a 'perfect simmer'). Thus two tools are required each with its own
definitions. Ant WBT performed using validated tools would provide a valid
result - for any stove-pot-fuel combination and can be reported as such.

Because stove programmes are becoming more and more dependent on field
observations (actual performance) we can expect more pressure to develop
valid tools that will work in a wide variety of circumstances. Indeed we are
under that pressure right now. The reason is people are tired of funding
failed stove programmes. They fail because a) people don't want to use the
products or b) the product does last long enough to be worth buying or c)
the product does not perform as claimed or d) the product is not flexible
enough to cook the foods and perform the tasks needed.

If the test is not culturally relevant it will not provide meaningful
comparisons. The idea that hundreds of protocols will be needed is not,
however, correct. Scientific tools are already provided for normalising
results.

>How would newly developed procedures supplement, partially replace, replace
existing protocols?  Of course, this is also related to the goal for
harmonization as raised in the IWA.  

Whatever process is used, the methods used in the past have not provided
useable tools. That is why I am awake long into the night corresponding with
other practitioners trying to rush to market tests that are valid for the
circumstances were we are working. We are frequently asked for 'WBT' results
and 'the real answers'. That is an indictment, not of our motivations, but
of our achievements to date. It is a bit embarrassing!

>Paul also raised the issue of recalculating older test results so they can
be comparable or harmonized.  I think another issue to consider is how much
have technologies changed and are the older test results still applicable.  

My concern is that the Stove Inventory is a chart of performance measured
with tools that provide dramatically different results for the same stove. I
am not exaggerating. If the use of the output number is to trade Carbon
offsets (and it is) then reporting that a 19% fuel efficient stove is 54%
efficient constitutes malpractice. That is the view of the people paying the
piper.

>We are also planning discussions to map out a strategy for integrating
future testing results and data together, especially to communicate stove
performance (IWA Tiers) that has been independently evaluated.  

That is strongly seconded by me!

>Part of the discussion should be around what is the minimal set of data
that the testing centers need to share to facilitate collaboration and
harmonization among the different centers.

I can open that conversation by suggesting that if a claim is publicly made
the raw data (in the form of the reporting spreadsheet on which the
calculations were made) is also made public. It may be possible to limit it
to that information needed to recalculate the result based on what we now
know are problem areas (like the treatment of fuel moisture, charcoal
remaining, water volume processed, averaging procedures, conceptual
considerations and procedural relevance.

Having that information would a) allow us to recalculate the old WBT's in
the Stove Inventory into a modern form, b) determine what the IWA tiers
would be if they were set using current understanding, c) permit
mathematically correct averaging of multiple tests to give and 'average
performance', and d) permit the validation of tools that can be incorporated
into an ISO Standard of stove testing methods.

Best regards

Crispin

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.bioenergylists.org/pipermail/stoves_lists.bioenergylists.org/attachments/20130406/cbeb34ef/attachment.html>