<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=koi8-r"><meta name=Generator content="Microsoft Word 14 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0mm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoPlainText, li.MsoPlainText, div.MsoPlainText
{mso-style-priority:99;
mso-style-link:"Plain Text Char";
margin:0mm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Balloon Text Char";
margin:0mm;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";}
p.MsoNoSpacing, li.MsoNoSpacing, div.MsoNoSpacing
{mso-style-priority:1;
margin-top:5.0pt;
margin-right:0mm;
margin-bottom:5.0pt;
margin-left:0mm;
mso-add-space:auto;
font-size:11.0pt;
font-family:"Arial","sans-serif";
color:#1F497D;}
p.MsoNoSpacingCxSpFirst, li.MsoNoSpacingCxSpFirst, div.MsoNoSpacingCxSpFirst
{mso-style-priority:1;
mso-style-type:export-only;
margin-top:5.0pt;
margin-right:0mm;
margin-bottom:0mm;
margin-left:0mm;
margin-bottom:.0001pt;
mso-add-space:auto;
font-size:11.0pt;
font-family:"Arial","sans-serif";
color:#1F497D;}
p.MsoNoSpacingCxSpMiddle, li.MsoNoSpacingCxSpMiddle, div.MsoNoSpacingCxSpMiddle
{mso-style-priority:1;
mso-style-type:export-only;
margin:0mm;
margin-bottom:.0001pt;
mso-add-space:auto;
font-size:11.0pt;
font-family:"Arial","sans-serif";
color:#1F497D;}
p.MsoNoSpacingCxSpLast, li.MsoNoSpacingCxSpLast, div.MsoNoSpacingCxSpLast
{mso-style-priority:1;
mso-style-type:export-only;
margin-top:0mm;
margin-right:0mm;
margin-bottom:5.0pt;
margin-left:0mm;
mso-add-space:auto;
font-size:11.0pt;
font-family:"Arial","sans-serif";
color:#1F497D;}
span.PlainTextChar
{mso-style-name:"Plain Text Char";
mso-style-priority:99;
mso-style-link:"Plain Text";
font-family:"Calibri","sans-serif";
mso-fareast-language:EN-CA;}
span.BalloonTextChar
{mso-style-name:"Balloon Text Char";
mso-style-priority:99;
mso-style-link:"Balloon Text";
font-family:"Tahoma","sans-serif";
mso-fareast-language:EN-CA;}
span.EmailStyle22
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle23
{mso-style-type:personal-compose;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-CA link=blue vlink=purple><div class=WordSection1><p class=MsoPlainText>Dear Dean<o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>The tests were performed using the WBT 4:<o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>“In this study, twenty-two cookstoves burning six fuel types (wood, charcoal, pellets, corn cobs, rice hulls, and plant oil) at two fuel moisture levels were examined under laboratory-controlled operating conditions as outlined in the Water Boiling Test (WBT) protocol, Version 4.”<o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>You said: “You can see that fuel use is measured, not assumed, etc.”<o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>I beg to disagree. The fuel use is not measured, it is calculated from the energy available from the mass of fuel burned, compensated for unburned char. I have just had confirmation of this from Jim yesterday morning. In other words, he followed the experimental procedure laid down in the WBT protocol exactly. The spreadsheet does not have a place to enter the amount of raw fuel needed to accomplish a cooking task. That is why the results of the Quad 2 stove test (636 dry g) are so different from the actual fuel needed to perform a 5-litre WBT (1550 dry g). <o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>Rocket stoves, which also produce quite a bit of char, are credited with a fuel consumption number that is not reflective of their actual performance. That will probably interest quite a number of organisations. It may explain a lot of the difference between WBT, CCT and KPT results. Maybe not – there are several other problems with the protocol and calculations.<o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>You have many times referred testers to the CCT for field confirmation of <i>actual</i> fuel use and warned people not to rely on the WBT results. The CCT makes the same type of calculation for ‘dry fuel consumed’ and does not determine the amount of fuel the stove actually draws from the forest for each cooking event. I checked the formulas. Like the WBT, it too finds out the approximate amount of heat that could be generated from the fuel and divides it by the dry heat content of the fuel type used for that test. <o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>The calculation method assumes that charcoal produced is actually raw fuel not consumed which leads to a significant misrepresentation of the performance with certain stove types, generally crediting them with using much less fuel than they do and over-rating the overall thermal efficiency (but still giving a proxy for the heat transfer efficiency).<o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>‘Overall thermal efficiency’ is not the same as the heat transfer efficiency. The WBT 4.1.2 tries to measure the HTE (see Fig 2 in the paper) but the paper calls it the OTE. I understand the ‘overall efficiency’ as being the amount of raw fuel needed to feed the stove to accomplish a heating type task (like heating water). There are many industrial examples of this. In the paper it is labelled ‘overall thermal efficiency’, but is it actually a proxy for the heat transfer efficiency with several losses uncounted (as can be seen by the way the number is generated).<o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>>And, as you can imagine, in this EPA peer reviewed study the statistical significance is good.<o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>Again, I beg to disagree. The statistical significance (or rather, the confidence we can have in the result) is undermined by the use of an averaging technique that is not accepted by statisticians as valid. How can a test method (and then the derived test results obtained by using that method) be ‘statistically significant’ if the comparison of individual test results is only made after grouping them first? This demonstrates that the peer review is only as good as the reviewers. If the reviewers do not themselves review the test method, or do not ask that the author provide the results of an independent review of the method, you ‘get what you pay for’. Even then, an independent review might be challenged if it contains its own mistakes. These things happen all the time.<o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>A peer reviewer might have noticed, for example, that the heat value of rice hull char was credited with a heat content of 29.5 MJ/kg. This is far from the actual heat content which is closer to 12-14 MJ (an error of —100%). <o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>Because the fuel use number is calculated from the net heat value of fuel burned (according to both me and Jim) and because the rice hull char remaining has a far lower heat content than the number applied (according to any reference you care to find) the stove was credited with a significantly higher thermal efficiency than it actually has. This resulted (because of the erroneous manner in which the fuel consumption is determined) in a much better fuel consumption rating than it actually has for two reasons instead of the usual one: 1) the error in the heat value for the char remaining, and 2) the error of not measuring the amount of raw fuel the stoves consumes to perform each test.<o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>If the Journal provides us with the spreadsheets from the actual tests as supplementary material for the published article (normally required for publication), we will be able to reproduce the work mathematically and see what the effect of making the necessary corrections would be. I expect there would be a significant change to the ratings given to all char-making stoves and all those burning rice hull. How much? We can’t tell until we see the supporting documentation for the article. My guess? 50-100% increase in fuel consumed per WBT and a halving of the number for the overall thermal efficiency. That supporting documentation will also include the version of the WBT (might be v.4 or 4.1 or 4.1.1 or 4.1.2) so we can reproduce it exactly.<o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>The published article system is where the real progress comes in science and there is no problem with challenging the content of peer-reviewed, published material. Published articles are a form of discourse. If something is published without the supporting material required to replicate the result, including calculations, it is not normally considered to have entered the published domain and remains ‘grey literature’. <o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>There are scores of grey literature articles, monographs and pamphlets claiming that the WBT, CCT and KPT are good tools for developing and determining the performance of stoves. What we have never seen in a Journal is a review of the procedures and mathematical methods conducted by an independent lab to put some confidence intervals on these WBT results. Dr Penn Taylor’s Masters and PhD theses have not been published. They address the issues and give a precision of about 50% for the WBT 3.1 (which was under review at the time). The accuracy was not determined. <o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>Berkeley’s review of hundreds of old WBT 3.1’s may be perfectly valid as a method, but it’s results rest on the platform of the validity of those tests for which there is yet no independent confirmation and very real suspicion that the results are not representative of true stove performance from an efficiency or fuel consumption point of view. The frequent warning by Aprovecho and others not to rely on WBT results (it is even written into the IWA is it not?) is itself enough to view with suspicion anything based on a review of hundreds of them made using an old, uncorrected version of the UCB test, not so? That is what Tom Miles was asking. What exactly were they trying to show?<o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>Regarding confirmation of more recent tests, there is a bit of a problem with the WBT 4.1.2 because it appears to be getting some undocumented changes made anonymously. I wrote to several people asking who is actually controlling this (signing off on suggested changes). I have no name yet. The reason I asked is because the current 19 Jun 2012 version has at least two changes that were not present earlier in the month but the version number is the same. <o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>One change I noticed is that the order of the test sheets reads Test 2, Test 1, Test3 instead of Test 1, Test 2, Test 3. The other is the introduction of an error in the formula in cell i30 of the Results tab that materially affects the CoV for the burn rate which may lead to a set of tests being considered valid when they are not (as far as some labs are concerned – not all labs use the CoV). Jim Jetter mentioned another one so that is three just on that version.<o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>Until the process of correcting the WBT is formalised the sorts of problems described above and in previous communications will continue to dog the stove community. In the meantime it is perfectly reasonable for stove projects to develop and use alternative test methods that are scientifically sound, properly documented and independently validated.<o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p><p class=MsoPlainText>Regards<o:p></o:p></p><p class=MsoPlainText>Crispin<o:p></o:p></p><p class=MsoPlainText><o:p> </o:p></p></div></body></html>