[Stoves] "Those of us who believe that the WBT is critical to stove improvement"

Sun Dec 17 01:06:37 CST 2017

Dear Nikhil

In answer to questions below and earlier questions you posed about the WBT’s ethnicity:

The WBT was created in it’s modern form by Berkeley, a contract between Shell and Kirk Smith. It had no ‘owner’ and Aprovecho, which was involved originally, continued to use and promote it. They made changes in 2007 in collaboration with at least one of the original authors (Rob Bailis). That is the origin of WBT3.1. It did not have an official ‘owner’ or more correctly, a ‘custodian’ for years. Kirk Smith continued to use WBT 3.0. Everyone in the US-based groups used CCT 2.0 which has formulae that divert from WBT 3.1. I don’t know why anyone is surprised when there are ‘differences between lab and field’ when a perfect match would be reported differently anyway.

WBT 4 was created by a technical committee of ETHOS – their first ever formed. It was supposed to be a collaboration between supporters and objectors to WBT 3.x’s numerous problems. Forming the group was proof enough that there were recognised issues. As it happened, the process was taken over by and taken into secret by a portion of the whole and WB4.1 was the result. It retained many of the problems of the earlier version including inappropriate metrics and incorrect calculations. It could not be corrected ‘officially’ as the original group formed to create it never re-convened, but WBT 4.1.2 appeared anyway. It did not fix the main problems.

As a result of this recalcitrance, GIZ, then GTZ, started to use methods developed with the large ProBEC Programme (14 countries) developed from first principles because there was a lot of product development going on in the SADC projects under that programme. This evolved into a ‘test method’ and that later was developed at the SeTAR Centre at the University of Johannesburg. In Africa we call this UDI, meaning Unilateral Declaration of Independence.

Even though the WBT 4.1.2 had no official custodian, it was referred to in the IWA 2012:11 as a ‘valid test method’ though in who’s opinion was not stated. No published version of it had been peer reviewed – not that I can find. It was available for download from PCIA’s website (Partnership of Clean Indoor Air) under the EPA.  It was never accepted as an EPA test method (because they have no mandate to deal with cooking stoves) and because it would not have survived any technical review process.

When the GACC was formed, they advocated the use of the WBT 4.1.2 but without a custodian there was no one to appeal to to make urgently needed corrections. At a large meeting of interested parties (more than 40) at the EPA stove lab in North Carolina. The problem of making corrections, the need for which was by now generally accepted, was highlighted and discussed. It was proposed by Prof Harold Annegarn, then head of the SeTAR Centre at UJ, proposed that the custodianship be taken on by the GACC.

He noted that we cannot even discuss using a test that has no official version, no mechanism for improving it, and no one in charge of hosting its official version and maintaining a detailed record of changes. All these are necessary to have and maintain a standard test method.  Ranyee, head of testing, agreed on behalf of the GACC and shortly thereafter the link to the documents (the test method and the accompanying spreadsheet) was moved to the Test Methods section of the GACC website.

The major question is why was it is so important to hold on to WBT? Where were the vested interests? Could it be that changing the stove test protocol for EPA's ongoing work would render future results incomparable with the past ones? Of course. It is well known that Berkeley, in for form of Kirk Smith, Dana Charron, Berkeley Air, the stove testing group, the students working on related activities including air quality and stove testing, wanted to continue to use the version developed there in 2003 (Shell) or one of its close derivatives.

If the major conceptual errors and miscalculations of the WBT were to be ‘publicly corrected’ it would certainly place a question mark over a large number of test results that had been used to support policy and project decisions over a number of years.  The reasons are easy to find: the errors are so fundamental that it would at the least require a recalculation of previous results. The conceptual error of claiming an efficiency for simmering, for example, would have to be replaced with something with a physical basis. The IWA decided on additional conceptual errors, more than WBT 3.0 because it introduced three new invalid metrics for low power that had not even been proposed to whoever controlled WBT 4.1.2 at the time.

In this chaos, the SeTAR Centre continued to develop its test method including conducting hundreds of tests used for developing products and ‘testing the test’. It was introduced to Mongolia for the development (by GTZ) of what is now the KG4 series of high performance crossdraft space heating and cooking stoves. The formal inclusion of community behaviour research into what was by then called the Heterogeneous Test Protocol (HTP) to produce the CSI Test Protocol (which includes a stove testing method).

Fabio Riba showed conclusively that the WBT stats are inappropriately conceived and functionally useless. Extensive replication of tests using WBT 4.x at the EPA lab supported their conclusion that even 100 replications would not provide a level of confidence that met the IWA requirement of 1/3 of a tier span. No national authority would be willing to conduct 10 replications let along 100 so the WBT is functionally useless.

Nothing in the published literature on the problems with the statistics is found in the ISO document. It was written as if simplistic averaging could somehow compensate for a fundamental failure to understand the problems: stove performance results are not normally distributed, and the problems of making claims based on a small number of tests (5 or less).

Who was harmed by the WBT? The entire cooking stove industry. Every donor, every stove recipient. Through ignorance or design, the WBT has been impressed upon the stove programmes outside China and India. India’s test is hardly better: though it does a creditable job on the cooking efficiency metric, it is worse on contextuality (because of the fuel and fueling procedure used). The Chinese test is very simple and contains only one mathematical error (double counting the energy gained by the pot between 95 and 100 C).

Several papers from the China Agricultural University’s Renewable Energy Research group in the College of Engineering have been written comparing the results obtained by test methods from India, China, Cambodia, South Africa and the WBT.  Once corrections for different conceptual errors were made, the methods of calculating the fuel efficiency were in significant agreement, save for the Chinese test which consistently produced higher values across the board. We find this very interesting. The Chinese test is intended for high mass stoves – certainly higher than the WBT’s typical targets which in China are called ‘Picnic Stoves’. The impact of the high mass is reflected in the Chinese result which credits simmering without no fire.

It may well be that with other test protocols we might get more reliable efficiency estimates, I don't know about PM2.5 estimates - or whether the WHO Tier 4 target set with a fatally flawed test method and seriously compromised "single box" air circulation model can be met at all.  Shouldn't we worry about exposures rather than hourly average emission rates?  As soon as one accepts that is the real problem to quantify, the stove test is distinct from it.  There are 4 typical types of Indonesian kitchen structures in Central Java. No single air circulation model can represent them. An emission rate per unit time is not very meaningful. An emission rate per unit of work done could be supported in principle because it is the pollution created per unit of work done. To know if that means anything to a real cook, one has to know first, for example, if that family produces palm sugar 8 hours a day because that would potentially be 11 hrs of exposure, not 3.

This PM2.5 issue be relevant to char-making stoves - if a stove really does a great job of making char and burning off all the pyrolytic gases, its promoters will want to pick a protocol that both gives dependable results and reports the char production on a mass or energy basis in an acceptable manner. There is no point in creating yet another protocol or metric that, right from the outset, is fairly criticized for being conceptually or mathematically defective.

‘An agenda’ cannot trump scientific fundamentals. I think we have adequate proof of that.

Regards
Crispin

++++++++++

Xavier:

If there is no harm alleged, who would bother? I did not ask you for a proof, merely an allegation that you can state as an allegation. Everybody is entitled to an allegation based on reasonable knowledge.

But whose problem is it that WBT gives results that are unreliable? Where did you ever see a piece of regulation that says cookstove testing protocol has to give reliable results?

As I have told you, WBT of the American variety has no legal authority. It's a toy EPA contractors borrowed from earlier USAID contractors and kept revising it the way they wished, to obtain measurements for some metrics of their interest - fuel efficiency, CO, PM2.5 - in order to publish their papers in peer-reviewed journals.

Since all the "peers" of interest were familiar with WBT, they likely went along with convenient pretense of science.

I was told - I don't remember by whom, maybe I read it somewhere - that EPA made GACC - i.e., the UNF, an EPA contractor that is an opaque, unaccountable pretentious private agent in the business of making money in ways blessed by convenient peers - the "custodian"of WBT.

What is UNF comptence, and what is EPA's jurisdiction, for this transfer of custody?

None, and none was required because all that EPA wanted to do was spend some research funds on US contractors.

The Emperor Pretending Authority has no clothes. EPA is lawless. It has gone rogue.

Which is convenient because its contractors can keep on publishing results that WHO then uses to cook up the Global Burden of Disease from Household Air Pollution, see for example here<https://nam03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fehjournal.biomedcentral.com%2Ftrack%2Fpdf%2F10.1186%2F1476-069X-12-77%3Fsite%3Dehjournal.biomedcentral.com&data=02%7C01%7C%7Cd6ce6c94c24d45c400d208d544416dcd%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636489970155240058&sdata=ZYlJIs9vBvrMeoOOK1jUzoCw4%2BnCjkq93OBipL3M%2FfU%3D&reserved=0>.

Then comes the real monkey business of "international standards". Some key documents in this exercise - here<https://nam03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpubs.acs.org%2Fdoi%2F10.1021%2Fes301693f&data=02%7C01%7C%7Cd6ce6c94c24d45c400d208d544416dcd%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636489970155240058&sdata=PbiSmypGu%2B5wGJw%2F1j9Vlgn5Q5Eh5PPbi9xefbMFcao%3D&reserved=0> and here<https://nam03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fehp.niehs.nih.gov%2F15-09852%2F&data=02%7C01%7C%7Cd6ce6c94c24d45c400d208d544416dcd%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636489970155240058&sdata=S9sdTtBH%2BIrOUzBrHJIyZds5Y7yhq50aO9MGXWGTMnQ%3D&reserved=0> - are based on WBT in the lab.

So, whatever was limited to merely academic matter of publishing papers was transferred to a policy-relevant exercise of IWA to TC-285.

You know I don't find the Lima Consensus, the IWA or the ISO TC-285 of any value in testing of stoves because I think the metrics themselves are at fault.

Whoever has a theory that higher efficiency or lower PM2.5 hourly emission rate has a causal predictability for all the alleged problems with solid fuels - resource depletion, disease incidence, climate change, and incidence of sexual violence - is merely parroting groupthink, a convenient ideology for raising money and spending it on rich theorists of rich theories.

There is no evidence that efficiency and hourly PM2.5 emission rates had anything to do with these ills in the past. Yes, I can attribute anything to anything and get away with it so long as my pals agree and we all parrot this attributability. But attributable doesn't mean causal and even if it did, the reverse causality may not exist.

So, to close the argument on this on my side: a) WBT doesn't matter, except to certain vested interests, and past WBT results are of no value when it comes to designing cookstoves for the future; and b) there is no theory of change in the IWA metrics and Tiers.  In any case, all this is a means to make money in the name of the poor. That is what Washington DC is suited for.

Now my answers to your question, between *** below.

•         There’s a loophole in the emission testing system of Volkswagen. It allows cheating. But, you say we have no way to clearly know how emissions are harmful to the population. So, we shouldn’t care if the engine is clean or not, and if the test is unreliable or not, because we don’t know the effects of emissions on health anyways.

So should we change the system or not? Should we ask Volkswagen to use a testing system that is reliable?

*** The comparison is invalid. There is a legally established authority for setting emission standards for auto vehicles in the context of legally set standards for ambient air quality. I have repeatedly told  you that what goes on in the name of "clean cookstoves" has no legal authority nor a theory of improving household air quality. There are no baseline data - other than spotty measurements collected in the WHO database of studies - for household air quality anywhere in the world. What WHO did is simply slap on assumed concentrations by age (up to 5 and 25+) and sex to all people supposed to be using solid fuels for cooking (which too is another model estimate with no data on quantity and quality of fuels consumed or emission rates). You don't want to acknowledge that all this is a charade, only to satisfy people engaged in it and snowed by it so they can keep publishing papers and making speeches to raise money. Including from Gates Foundation and HHS. I have yet to see any argument that suggests to me that I am on the wrong track here. ***

•         The NASA realize some of its calculations basically find that 1 + 1 = 3. Those calculations are actually present in many of their research projects, for various technologies. But they have no way to know how much impact it had on the projects, because the projects are still going, there are technologies being developed, all is seemingly quite fine.

Should the mistake be corrected or not?

*** That is for NASA to decide. Who finds the error and how? Either there is a whistleblower or the NASA Inspector General finds it. Once the information is in the open, anybody with a standing can take NASA to court if it doesn't correct the error. The point is, process matters. Legal authority to do something exists, and there is a framework for transparency and accountability. Just remember the shuttle disaster and how Richard Feynman showed to us all that the O-ring was not up to withstanding extremely cold temperatures. Here - in the case of EPA as well as its contractors such as GACC and ANSI, there is no legal authority - under US law, that is - for the use of WBT or anything else. I haven't even asked John Mitchell at EPA if there are internal evaluation report of its cookstoves program and contractors, but I don't need to. GACC is proof plenty of a lawless, opaque, unaccountable entity. Anybody in this game can do anything, and of course you too can complain; I strongly support your complaint. But to me, there is no point bothering about reliability of WBT in predicting efficiency with actual fuels (not oven-dried wood of a particular type, chopped in a particular way) and actual use (actual cooks preparing actual meals or heating water and kitchens). Who cares and why? I have yet to see anybody in the manufactured stove distribution business to claim that s/he relied on WBT to design and sell his/her stove. The reason is simple - they all know what a racket lab testing is, and rely on it to get some marketing edge or some money from carbon credits. S/he doesn't have to guarantee any performance in terms of fuel savings or disease incidence (which is where BAMG steps in to sell aDALYs). Not that users rely that much on promised fuel savings or can do anything if the price or collection cost of fuel changes. Stove/fuel choice is dependent on a wide range of factors, above all the cook's ease of adapting to new technology. But real cook and cooking has not been the interest of WBT faithful. I said as much 34 years ago and haven't found any reason to change my opinion. In short, this is an invalid comparison. You, sir, are taking the drama as if it were a real life event. No, it is not. What you are observing in TC-285 is, as far as I can tell (only from the first DIS), shenanigans. Won't amount to anything except cooking up excuses for Gold Standard and Goldman Sachs. There is no historical baseline for anything anywhere, nor a future baseline, and nobody is going to be able to measure actual performance except by spending loads of money on M&E consultants. Again, what I see is all about making money in the name of the poor. If it were ESMAP or World Bank or USAID, they are bound by certain process rules for spending public money and have the authority to do so. EPA and its contractors simply have no place in poor people's kitchens. They may all pretend high morals and noble intentions; I for one don't care what makes rich people richer but does nothing for the poor - not even move money. (I of course know GACC's projects on "improved" charcoal stoves. I don't know what lab test protocol they used, what promises of reduced disease incidence or deforestation they made to whom and over what period. All that can be made public for people to judge. When it comes to TC-285, only the final product would show. Whether it is used by anybody is anybody's guess. ***

I have an other question for both of you:

•         How do you measure the impact of the unreliability of the WBT? Do you have a methodology?

 *** Why should I care? I have told you the primary performance metric I have is usability, pleasing the cook enough for her/him to make a change, for whatever reason. Then and only then it makes any sense, for academic reasons if nothing else, to measure actual performance against some baseline performance. If there is no point in higher fuel efficiency or lower PM2.5 hourly emission rates, why should I care that WBT results are unreliable? Again, the limitations and pitfalls of WBT have been known for decades, and EPA contractors' generating newer and newer versions of WBT does not mean the fundamental conceit of metrics selected for experts' satisfaction. All this talk of M&E methodology for a check against promised "results" begs the question, what results does a cook want? We have this health fascism that no solid fuel cookstoves are "truly health protective" and that "stacking must not be permitted because the dose-response curve is non-linear at low levels of PM2.5".  It survives because there is big money behind it, not to speak of IHME/WHO deceit. Frankly, my dear, I don't give a damn. This is a cruel joke on my sisters and nieces worldwide. ***
No personal offense intended and I hope you don't take any. I agree something has to be done. To me, the first step is to stop an end to the TC-285 pretense. It is immoral.

Nikhil

On Fri, Dec 15, 2017 at 4:45 PM, Xavier Brandao <xvr.brandao at gmail.com<mailto:xvr.brandao at gmail.com>> wrote:

Dear Nikhil, Ron,

Nikhil, this is probably the third time you are asking me that same question:

« What are the proofs that the WBT did harm anybody? »

And my answer hasn’t changed, so I’ll tell you for the third time: « I have no proof, there are no proofs, because it has never been studied ».

Ask me that same question again, I’ll answer the same thing.

If you have 30.5 million dollars from the Gates Foundation to do a 5-year longitudinal study, please be my guest and lead the way.

And let’s wait for the report conclusions to contemplate if maybe we should act.

In the absence of that, we need to think carefully, and act carefully. But we need to act.

Nikhil, Ron, it’s my turn to ask you questions: