<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=us-ascii"><meta name=Generator content="Microsoft Word 15 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0mm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0mm;
mso-margin-bottom-alt:auto;
margin-left:0mm;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-CA link=blue vlink=purple><div class=WordSection1><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D;mso-fareast-language:EN-US'>Dear Dean and Kevin<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D;mso-fareast-language:EN-US'><o:p> </o:p></span></p><div><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D;mso-fareast-language:EN-US'>></span>Can you imagine a more thorough investigation than the international ISO process that is occurring? <o:p></o:p></p></div><div><p class=MsoNormal><span style='color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Yes I can. I think we should be discussing the concepts the underlie the purpose of testing long before writing international standards to capture them. <o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Kevin hits the nail on the head: the ISO process is an observed process, not a ‘good results process’. <o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>></span><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'># The first sensible step is to develop a scientifically valid testing procedure, which THEN would be submitted for ISO Approval. As long as ISO standards and procedures were followed, such a scientifically valid testing procedure would consistently give scientifically valid results.</span><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'> <o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Precisely.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>There are several aspects of stove performance that are not addressed at all that relate to the social acceptance and cultural acceptability of products. I invite Cecil to comment on this because he has been working on an evaluation matrix of social performance.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>The WBT, for all the assistance it has been for many years, is conceptually out of date on several levels. It does not ask the right questions. It cannot provide the necessary answers. It uses some questionable metrics and in general does not – in fact has not – delivered the programme results needed at this time in history.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>The pattern to date for the progress towards an international standard has been for the organisers to concretize the WBT as the default test method, to stand by it through thick, thin and noise, to promise future correction of known issues and to avoid serious discussion from first principles of what we are supposed to be doing. In particular the independent review of the WBT as ‘fit for purpose’ has not only been overlooked, it has been evaded. Ladies and gentlemen, this state of affairs is not going to fly.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>It is not (at all) necessary to take some existing ‘test method’ in order to have an internationally acceptable system for rating and ranking stoves. We have to solve fundamental technical issues first. I have noticed subtle changes to the text of communications referring to the development of an ISO standard that proclaim that the ‘Lima consensus’ which was 15 mostly Americans with a historical involvement in the WBT agreeing that the WBT was the right way to proceed. In fact they agreed on a certain <i>version</i> of the WBT. Is this correct? I do not want to misrepresent the decision. I was disturbed to hear that one comment made, when proposing the WBT as ‘the method’ was, “It is a good job Crispin is not here.” Why? Because I would have objected the fact the WBT has not been independently validated as fit for purpose, that some of the metrics are not scientifically validated, and that it (certainly at that time) gives wrong answers to questions it asks. Those are pretty serious shortfalls.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>I read lately that it was ‘international organisations’ that agreed to this method at Lima. I don’t think so. ‘International organisations’ were hardly in a position to make such an agreement without first having a ground-up examination of the needs for such a test and evaluation method. And that did not happen.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>It did not happen again when the IWA meeting was held. Again we were given (after a public comment period) the same old WBT with some edited metrics that failed not only to address the shortfalls of the old WBT’s but again avoided a ground-up re-examination of what we are supposed to be asking and delivering to the stakeholders. An examination of the WBT 4.1 as it then was reveals serious shortcomings both mathematically and conceptually. (In the interim most of the math problems have been addressed, but not the metrics or concepts. Everyone is trying to contribute to this, I believe.)<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>If the ISO process is a repeat performance of the IWA, I expect to be again shown a dressed up version of the WBT with the significant and fundamental questions still going unasked and therefore unanswered.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>The Lima meeting and the IWA meeting are being presented as having represented this much-needed consultation. They were not. They were brief and failed to address many technical points. That said, it does not address the missing portions of a meaningful test evaluation system. I am speaking of not providing an opportunity to examine the fundamental suppositions and the theoretical framework of a requested and supplied metric. Entrenching this in an ISO standard with a ‘Rosetta stone’ of conversions is impossible if there are fundamental gaps or defects in the concepts that are embodied in the ‘baseline’ measurement system.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>While I applaud the effort that is being applied to this matter – an internationally acceptable standard – there is a curious devotion to the WBT that is not justifiable. If we do not work out how to address fundamentals, there is not much point in putting on finishing touches.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>In short, getting a precise answer to the wrong questions is not helpful.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Regards<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Crispin<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p></div></div></body></html>