[Stoves] The need to continue the discussion: simmer efficiency

[Crispin Pemberton-Pigott]

Dear Jiddu

 

I am not convinced it is widely understood what the term ‘invalid metric’ means. People behave as if a metric has validity because the divided or multiplied some numbers which were arguably true to begin with. As Prof Lloyd says, if something is fundamentally wrong it cannot be edited or argued valid in certain circumstances. It is just invalid. 

 

An example of an unacceptable calculation in the WBT is the averaging of the cold and hot start high power thermal efficiency values. 

 

The underlying purpose of performing a cold and hot start test is because the results are expected to be different.  The claim is that it somehow compensates for stoves that have a high mass – an uninvestigated conceptual foundation. If such a determination were relevant, is that how to do it? The simmering already deals very well with retained heat as in the case of the Chinese biomass stove test which does a pretty good job of determining the influence of mass in their high mass stoves.

 

Let’s assume the WBT has the right intention and method, and look procedurally at the math of the results.  There would be no need to do the hot start if the result as expected to be the same. Correct? It is not done ‘as a check’ of cold start performance, it is done to find out the value for a hot start condition. For the moment don’t worry about relevance to cooking.

 

Each calculated value is an efficiency, which is a ratio. For the moment don’t worry about how the numerator and denominator for each efficiency are determined.  Here the critical failure is this: it is not mathematically permitted to ‘average ratios’ when the results are expected to be different. Similarly one cannot get the average thermal efficiency by averaging a large number of measurements each pair giving a ratio. 

 

One is not allowed to add the % values of 11/17th and 1/3rd by summing and dividing by 2 when the magnitude of the underlying elements of the formula are known to be or expected to be different.  This is not debatable. Doing so is fundamentally wrong because it doesn’t weight the values and doesn’t give the correct answer. It gives an answer I agree, but not a correct one and in many cases can be misleading.  

 

There was a discussion on this list of a guy ‘Alexander’ who Yuri helped me to find in Russia. He published a paper on this type of error in the Russian masonry stove test, a paper noted in the West by the Masonry Heaters  Association and Green Heat. You can’t get average performance by averaging values that are themselves averages of values with different magnitudes. It is fundamental to math and therefore to physics and engineering. Stove thermal performance is engineering performance.

 

I took this up with Ryan when he was editing the WBT v4.2.2 to make 4.2.3 and we discussed it by email for about 4 months. He unfortunately preferred the advice of someone else who apparently is unaware of the rules of high school math and his name is attached to the final product.  He left the incorrect formula in place so it is still there and it is still being done wrong and everyone is being given incorrectly efficiency values.  It may or may not hide important information.  (I will look at the concept of hiding important information when we come to discuss ‘tiers’ which must surely also be done.)

 

Conceptual review:

 

Looking at the efficiency averaging problem conceptually, we have an energy to boil value, twice. The work done is ‘boiling water’ with at least theoretically, the same change in enthalpy in the pot.  I say ‘theoretically’ because the value is normalised for temperature rise so it means the intention is to create a common value for ‘work done’: heating and bringing to a boil a known quantity of water. The total energy change, defined as the heat gained, is the work done.  Whether the WBT does it correctly or not is a different issue. The intention is to create a reporting metric that measures average useful work under two conditions.

 

The energy needed to heat the water in the pot is affected by the thermal mass of the stove. The heat that gets into the pot is not, and this is an important point.  The approximate amount of heat into the pot for both cold and hot start is about 1.7-1.8 MJ every time. The ‘boiling water’ condition takes the same amount of heat. The heat from the burning of fuel varies depending on how much heat is stored in the stove. This is obvious.

 

Cold Start

A = heat gained by the pot. B = heat from the fire, of if system efficiency is desired, B = the heat available in the fuel consumed from a source outside the system being investigated.

 

Hot start

C = heat gained by the pot. D = heat from the fire, of if system efficiency is desired, D = the heat available ibn the fuel consumed from a source outside the system being investigated.

 

Cold start efficiency = A/B

Hot start efficiency = C/D

 

Because we know in advance, and expect, that the values of A and B are about the same, and the values for B and D are different, it means that the averaging of the two efficiency numbers is not (A/B+C/D)/2 which is what the WBT uses.

 

The correct formula is (A+B)/(C+D) if you want the average performance for these two different conditions.

 

Fractions with a common numerator are averaged using the harmonic mean <http://mathworld.wolfram.com/HarmonicMean.html> , not the simple average. This is a rule of math and is used (in principle) every day to calculate, for example, the value of capacitors in series.

 

1/CT = 1/C1 + 1/C2 + 1/C3…

 

The harmonic mean of two values is a special case and is 

 

H (X1, X2) = (2 X1 X2)/(X1+X2)

 

Every time one calculates the performance in a stove using the WBT in the lab, the result is an incorrect value for the thermal efficiency (ignoring all the other issues – just because of the formula applied). How the thermal mass affects the result is still unknown because we already know the calculation is not being done correctly. 

 

What else can I say about this? This has been brought to the attention of the stove community for years. 

 

The WBT results do not predict performance in the field because the burn cycle is not similar to cooking, and the burn cycle affects the performance (a lot). Many of the calculations are not done correctly, are conceived incorrectly in the first place, and/or are not well chosen to represent performance, which is the whole purpose of testing. 

 

I think Cecil pointed out already that the Rocket Stove is designed, following thousands of tests, to optimise the calculate outputs of the WBT, not to perform well in the field under real cooking conditions.  We are not going to pretend that the WBT and Rocket Stove are not entwined.  Within the limitations of the design concept, it is optimised to give ‘good’ WBT numbers.  The same is true for some other stoves but not all.  The idea that they should is embedded in the ‘performance tiers’ of the IWA which requires that results determined using other methods be converted to WBT-equivalent results.

 

Suggesting that the WBT is ‘good for designing stoves’ is simply not supported by the evidence.  The advice in the text is to find the ‘real performance’ using another method. Then there is little value to be had from the WBT. No one can ‘design a stove’ using a set of measurements that don’t accurately represent performance.

 

Whenever a stove is rated by the WBT, and then by another test that doesn’t have all those errors, the results are significantly different. That is why I do not use the WBT for designing stoves. As I was saying to Julien, a stove designers test usually makes very specific measurements not an overall assessment, though of course it would have value as a predictor of performance if it was a representative, contextual burn cycle does using fuels and pots that are realistic.

 

A stove ‘designed’ using very dry fuel is very unlikely to perform well using damp fuel. That is why the Indian national test is a poor predictor of emissions performance. It uses totally dry fuel. One would have to change the stove architecture a lot to burn it well. Should people design to perform well in real life or to pass some national test, as you have pointed out?

 

The WBT burn cycle is locked in the method and the fuel is fixed and arbitrary. The result is first mismeasured, then miscalculated and expressed using questionable metrics. It is not merely that I have to avoid the WBT, I have a responsibility to the stove community to recommend against using it so people are not misled, an obligation if people are paying me for my opinion.

 

I am hardly alone on these condemnations. I already cited the IEEE paper by Yixiang Zhang et al.  Prof Annegarn, after reviewing the WBT said that its outputs were ‘not usable for any purpose’.   Prof Lloyd, who is chairing the ISO Working Group 1 for conceptual review said years ago it gives ‘irreproducible results’.  Many testers agree, and the proof is in the large differences people get when testing the same stove at different labs. The GACC sponsored some round robin testing. What were the results? Let’s check how that exercise went. Was it a test of the stove of the lab?

 

The insistence on using the final water mass three times in the calculations results in higher variability in the outputs than exists in the raw data – a sure sign of a conceptual error.   The Aprovecho Performance Targets (which are for some reason now included in the WBT results page) compounds these results.  If the measured performance on a variable changes by 5% the WBT performance should changes by 15%.  The Aprovecho improved stove metrics (which are not part of the WBT) can change by much more.

 

This variability has been addressed by Marcelo in his new paper which shows that the WBT has an inherent variability in the results of about 30% which means it is attributable to the protocol, not necessarily the stove.

 

Fundamental problem:

 

You can’t place stoves on tiers separated by 20% using a test method that has an inherent variability of 30%.  This matter was addressed during the IWA discussions and a requirement for repeatability embedded in section one of the document.  The WBT does not meet this requirement. Maybe no method does. We have to check first, not after.

 

Marcelo did not look into why the WBT has such a high variability. I have, as have others. It is directly attributable to the selection of the metrics, the variables and how the outputs are calculated.  Everyone is blaming the operator by showing that consistent results in good agreement ‘are possible’. That is not helpful if no one else can get the same result.

 

Low Power Specific Fuel (or Energy) Consumption

 

With reference to the low power metric for specific fuel consumption (simmering in another name) Dean has suggested that we could divide by the initial water mass in the pot to overcome the problem if using the more variable final mass. 

 

As the initial water mass is the result of what happens in the high power phase, the initial mass varies. You correctly point out that using a fixed number or a slightly varying number or a final highly varying number will make no difference to the calculation of an invalid metric. It is not invalid only because of poor variable selection.  The metric itself is invalid, so calculating it using different input variables will not ‘fix it’. It is not even broken. It should not exist in the first place. I think there is a poor realisation that there is such a thing as an invalid metric.

 

Further, we cannot say that, “Using the WBT can produce a ‘better stove’ than an open fire therefore it is OK for now.”  It is not ‘OK for now’.  For stick burning stoves it is might be than nothing but it certainly can’t deal with charcoal and pellet stoves and planchas and TLUD’s and who knows what else. These ‘starting gate failures’ matter. 

 

For char-making TLUD’s it is worse than nothing because it positively lies about the performance.  It claims that stoves using more than 1300 g per burn cycle are using less than 650.  It ignores ignition emissions. The list goes on and on.  We had those discussions right here. This misrepresentation is untenable. People are trading money on the basis of these false claims. I am not going to stand by and say nothing.

 

This is important and we have to deal with it now. Thanks, Jiddu, for contributing.

 

Regards

Crispin

 

 

From: Stoves [mailto:stoves-bounces at lists.bioenergylists.org] On Behalf Of Dean Still
Sent: Friday, February 20, 2015 08:39
To: Discussion of biomass cooking stoves
Subject: Re: [Stoves] The need to continue the discussion: simmer efficiency

 

Dear Paul,

 

The WBT is usually used for comparisons of performance because the results are internationally comparable. 

 

You probably can't use the CCT or KPT because there are many confounding variables.

 

The multi-national experts involved in the ISO process have to figure out a fair way to compare

 

performance that will be useful to consumers like an energy rating for a refrigerator or a miles per gallon sticker on a car.

 

I certainly don't have any idea what will emerge.

 

Good to have lots of folks involved! 

 

Aprovecho mostly uses the WBT/CCT combination. Each for a different and important purpose.

 

Best,

 

Dean

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.bioenergylists.org/pipermail/stoves_lists.bioenergylists.org/attachments/20150220/1e913795/attachment.html>