[Stoves] Advocacy action: ask the GACC to stop promoting the WBT (Frank, Xavier)
Frank Shields
franke at cruzio.com
Tue Apr 18 19:01:49 CDT 2017
Stovers,
Agree - the Round Robin testing is to test the labs. The labs actually determine the answer by plotting results on a curve. The outliers got the answer wrong because they did not do the test as instructed. To do a RRT on stoves all six boxes (6-Box system) must have instructions to be followed. This group is IMO has drafted way off course and we have no way of conducting a RRT program.
The WBT is exactly the test we need (once it is modified to work) and other like tests based on the same concept should be developed.
We have more than enough garbage data to play with and having more is not the answer. First we need to construct the test properly and then just a few data points is enough. Using biomass from each location the stove is to be marketed and certified labs that are at the top of the bell curve during RRT doing the tests we determine the best stoves. So simple. Its already being done with compost , soils and like matrices where there is no ‘real’ answer just the real answer the group of certified labs come up with for comparison purposes.
Regards
Frank
> On Apr 17, 2017, at 11:49 PM, Traveller <miata98 at gmail.com> wrote:
>
> Frank:
>
> Round Robin is not really a test. It is in a way comparisons of actual tests under a given protocol. Considering the primary conceptual and methodological limitations of WBT, I don't think RRT adds any value to the debates on protocols, performance matrices, or usability of stoves. It is a way of assuring selected experts.
>
> Xavier:
>
> Let me venture a guess as to where your analysis and expectations may have gone wrong.
>
> You demand too much of GACC. It inherited the PCIA from EPA and with it both the mindset of ETHOS and US stove clusters and a mandate to use WBT results to produced the catalog of "clean stoves" plus blessing VITA WBT in the IWA.
>
> Given that background, it is EPA, not GACC, that is indirectly responsible for persistent use of WBT. I suspect USAID, the other partner of PCIA, would have been more open to criticisms of WBT and accepting other test protocols.
>
> I accept Ranyee's clarifications; they are in line with what I had expected. When she says, "The biggest challenge now is that there isn't enough data to make many conclusions." I can cynically say that this was the case with WBT itself. Tami told us that WBT was merely a placeholder at the time of Lima Consensus.
>
> Things evolve slowly. You may reasonable ask why from Lima to the Hague, there was no official declaration that WBT is just one of the test protocols, why Regional Centers used WBT for which investment projects, whether WBT was appropriate for PM2.5 emission rates, but it is unreasonable to charge GACC was working on "something else". Because we do not know what it is working on - e.g., the DfID project on Evidence Base.
>
> I for one think there will never be enough data to make conclusions of the type warranted. All I ask for is evidence of usable stoves with demonstrated cost savings in field tests using whatever protocol.
>
> I suggest you gather together all your previous questions that you feel have not been answered and start the next round of discussions. I do support your view that GACC needs to put out Ranyee's answer in the general public domain - ""We all recognize that there is room to improve, and that is already the starting motivation for ongoing work by many people. There are protocol improvements that are in progress and in discussion, which will be published as soon as they are complete." - with an explicit declaration that GACC was never insistent on WBT being the only protocol or even a reliable protocol and is not suggesting so now.
>
> The ISO process is limited to recognized experts from individual member states, with some extraneous observers as approved by the TC chairs. It is designed to exclude people like you and me. We are not sanctified "experts", and ISO doesn't owe us anything. It is EPA and ISO TC that are responsible for what has happened in the past seven years, not GACC.
>
> Nikhil
>
> ---------
> (India +91) 909 995 2080 <tel:+91%2090999%2052080>
>
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Thu, 13 Apr 2017 21:19:25 +0200
> From: Xavier Brandao <xvr.brandao at gmail.com <mailto:xvr.brandao at gmail.com>>
> To: Discussion of biomass cooking stoves
> <stoves at lists.bioenergylists.org <mailto:stoves at lists.bioenergylists.org>>
> Cc: Ranyee Chiang <rchiang at cleancookstoves.org <mailto:rchiang at cleancookstoves.org>>, Leslie Cordes
> <lcordes at cleancookstoves.org <mailto:lcordes at cleancookstoves.org>>, Neeraja Penumetcha
> <npenumetcha at cleancookstoves.org <mailto:npenumetcha at cleancookstoves.org>>
> Subject: Re: [Stoves] Advocacy action: ask the GACC to stop promoting
> the WBT
> Message-ID:
> <CAJiUTNev_=G0WOWhZ55P5Vx4O07_Yj50Dd00diPhWcVAath09g at mail.gmail.com <mailto:G0WOWhZ55P5Vx4O07_Yj50Dd00diPhWcVAath09g at mail.gmail.com>>
> Content-Type: text/plain; charset="utf-8"
>
> Dear Ranyee,
>
> It seems neither you nor anyone at the GACC wish to reply to the questions I sent in my previous emails. It is a shame, because I think these are simple, straightforward questions, and they are simple to answer.
>
> I understood the GACC was committed to the highest standards of transparency and accountability.
>
> But there exist little to no information about the round robin testing, nor how the GACC plans to address the many issues related to stove testing, issues raised by numerous studies. I read the ETHOS presentation about the round robin testing, it leaves most of my questions unanswered.
>
> There is an urgent need to talk about these issues, work on solutions, and again, this has to be done in other spaces than just the ISO TC 285. There is a need for a strong effort, and we are waiting for the GACC voice on that.
>
> Given what is at stake, policies for 3 billion people, openness is crucial.
>
> This is exactly what is very well said by a recently published article of Nature:
>
> http://www.nature.com/news/energy-scientists-must-show-their-workings-1.21517 <http://www.nature.com/news/energy-scientists-must-show-their-workings-1.21517>
>
> "Closed systems hide and perpetuate mistakes." it says.
>
> WBT mistakes have been perpetuated for years.
>
> In other sectors, things are moving. The mayors of Paris and London are pushing for new evaluation systems allowing for reliable information on car
> emissions, so we avoid something like what happened with Volkswagen:
>
> http://www.cnsnews.com/news/article/paris-london-seoul-grade-cars-based-emissions <http://www.cnsnews.com/news/article/paris-london-seoul-grade-cars-based-emissions>
>
> The ETHOS presentation about the round robin testing is here:
>
> http://ethoscon.com/pdf/ETHOS/ETHOS2017/Penumetcha.pdf <http://ethoscon.com/pdf/ETHOS/ETHOS2017/Penumetcha.pdf>
>
> According to this presentation, the objectives of the RRT are to:
>
> - "Facilitate collaboration to establish high quality testing and quality assurance procedures
> - Ensure consistent and reliable methods and results
> - Provide resources and tools to diagnose and troubleshoot issues in the future
> - Demonstrate potential for high-quality testing and evaluation services"
>
> We are not sure how exactly these goals will be achieved. We don't know where the RRT is starting from, where it is going to. Just like the previous communications about the ISO TC 285: we have an idea of how things are organized, but no idea of the actual content, of what is actually being discussed, and decided for the sector. This is a remark I made to you Ranyee, I don't know if you remember, long ago, about the GACC webinar on the TC 285, it might have been June 2015, I think.
>
> The goal of the RRT is to "Strengthen the sector (not focused on individual testing centers, products, etc.)".
>
> The GACC is acting for and on behalf of the stove sector, but I think the sector could have been more involved.
>
> I understand that the GACC is working with its 22 Regional Knowledge Testing Centers (RKTCs), but what I mean by involvement of the sector does not mean only being asked to provide data. I am talking about being truly consulted and involved in shaping the methodology of the study. To be involved in the decision-making process. It is about bringing different views about state-of-the-art testing, addressing real issues, and involving the top researchers on the matters, like the ones who wrote the studies I quoted earlier.
>
> I feel that instead of working on the issues brought by the studies, the GACC is working on something else.
>
> Correct me if I am wrong in my analysis or if I am missing something. But I am not sure how actually the RRT will help solve the questions we have. 22 RKTCs all around the globe will do Water Boiling Tests for 3 different types of cookstoves. A lot of data will be generated. But given the very high variability and uncertainty of WBT results, and the differences likely to occur in the way testing is conducted from one testing center to another (plus the data collected from other actors), how reliable will be the data collected? Is it comparable at all? Is it usable at all?
>
> And what about other test protocols?
>
> Since there is obviously very little will from the GACC to talk openly or to support legitimate and collaborative efforts to move forward, we will continue to discuss the matter, here and in other spaces. We will do so directly with the manufacturers, project implementers, large NGOs, researchers, testing centers, humanitarian agencies, funders, and various partners. We will keep openly critiquing testing, collecting scientific work on this matter, and will keep encouraging collaborative work. Anyone is welcome to join, anytime.
>
> Best regards,
>
>
>
> Xavier
> >
>
> ------------------------------
>
> Message: 3
> Date: Fri, 14 Apr 2017 04:23:39 +0000
> From: Ranyee Chiang <rchiang at cleancookstoves.org <mailto:rchiang at cleancookstoves.org>>
> To: Xavier Brandao <xvr.brandao at gmail.com <mailto:xvr.brandao at gmail.com>>, "Discussion of biomass
> cooking stoves" <stoves at lists.bioenergylists.org <mailto:stoves at lists.bioenergylists.org>>
> Subject: Re: [Stoves] Advocacy action: ask the GACC to stop promoting
> the WBT
> Message-ID:
> <MWHPR08MB2559D8B1D673EE2C6FDE10E9B3050 at MWHPR08MB2559.namprd08.prod.outlook.com <mailto:MWHPR08MB2559D8B1D673EE2C6FDE10E9B3050 at MWHPR08MB2559.namprd08.prod.outlook.com>>
>
> Content-Type: text/plain; charset="us-ascii"
>
> Dear Xavier and all,
>
> I thought that I had answered the questions already in my previous e-mails, but happy to clarify. I'm including the text of a message that I sent previously, with additional clarifications. I think it will also be helpful for everyone to remember we all agree that there is important work to do, so hopefully that can collaborative tone can come through in how we all communicate together.
>
> "We all recognize that there is room to improve, and that is already the starting motivation for ongoing work by many people. There are protocol improvements that are in progress and in discussion, which will be published as soon as they are complete." This means that we are not promoting any particular protocol. In fact, we provide links to multiple testing-related resources from many organizations<http://cleancookstoves.org/technology-and-fuels/testing/protocols.html <http://cleancookstoves.org/technology-and-fuels/testing/protocols.html>>. As soon as new protocols are completed, we will add those to the website list. And we know that all protocols can be improved, harmonized, and that this needs to be a group effort. And that is happening. The ongoing ISO process is designed to bring together experts with different perspectives into one discussion, and that has already led to quite a few updates that those not participating may not be aware of.
>
> "Many RTKCs also participated in a recent round robin testing exercise, with the results supporting how we improve testing methodology and how testing centers establish quality assurance plans. Thanks to the dozens of people who have been contributing their time and expertise over the last few years." The biggest challenge now is that there isn't enough data to make many conclusions. That's a limiting factor and also why there's such little detail in the presentation from ETHOS. When RTKCs came together to develop the Round Robin Testing plan, the agreement was that people would share data within this group as a way to help this group to improve. So that is our primary focus. There are ongoing discussions amongst the participating testing centers about the details of the results and implications. We are also still getting data from additional testing centers. The most helpful thing for others would be to try out the RRT instructions and add your data to strengthen the analysis.
>
> "We will continue to work with RTKCs to improve methods, testing, and data sharing. For those people who have not been as active in this global collaboration, we continue to invite people to contribute to the ISO discussions, to contribute data to the Round Robin Testing effort, and to contribute data to the Clean Cooking Catalog so that we can continue our analyses on the strengths and areas for improvement." If anyone would like to join the ongoing collaborations to improve protocols or using the protocols at testing centers, please contact me directly and separately from the listserv. Because there are so many conversations on the listserv and they're meant as group messages, it's easier to track personal e-mails that are sent to individuals. In particular, my suggestions for how to help improve testing protocols are to 1) join the ISO working groups<http://cleancookstoves.org/technology-and-fuels/standards/how-to-participate.html <http://cleancookstoves.org/technology-and-fuels/standards/how-to-participate.html>>, review the current drafts, and propose specific changes and 2) contribute data to the Round Robin Testing exercise.
>
> Best regards,
> Ranyee
>
>
> ------------------------------
>
> Message: 4
> Date: Thu, 13 Apr 2017 22:00:25 -0700
> From: Frank Shields <franke at cruzio.com <mailto:franke at cruzio.com>>
> To: Discussion of biomass cooking stoves
> <stoves at lists.bioenergylists.org <mailto:stoves at lists.bioenergylists.org>>
> Cc: Xavier Brandao <xvr.brandao at gmail.com <mailto:xvr.brandao at gmail.com>>
> Subject: Re: [Stoves] Advocacy action: ask the GACC to stop promoting
> the WBT
> Message-ID: <63B4A3E8-E79E-4396-AB32-B79EDCB834B1 at cruzio.com <mailto:63B4A3E8-E79E-4396-AB32-B79EDCB834B1 at cruzio.com>>
> Content-Type: text/plain; charset="utf-8"
>
> Dear Ranyee
>
> A round robin test program is where a single sample is subdivided and sent to different labs to have the same test protocol conducted. Then the results are sent to an outside person for statistics comparison. Is that what was done? You say there are dozens of people involved?
>
> I would like to be one of them if I have the equipment required to do the test protocol you specify. What is that protocol? How much sample do you send out? and where do the results go to? and who are the others involved in the Round Robin program?
>
> Thanks for this very needed work.
>
> I look forward to your reply.
>
> Frank
>
> Frank
> Frank Shields
> Gabilan Laboratory
> Keith Day Company, Inc.
> 1091 Madison Lane
> Salinas, CA 93907
> (831) 246-0417 <tel:%28831%29%20246-0417> cell
> (831) 771-0126 <tel:%28831%29%20771-0126> office
> fShields at keithdaycompany.com <mailto:fShields at keithdaycompany.com>
> ------------------------------
>
> _______________________________________________
> Stoves mailing list
>
> to Send a Message to the list, use the email address
> stoves at lists.bioenergylists.org
>
> to UNSUBSCRIBE or Change your List Settings use the web page
> http://lists.bioenergylists.org/mailman/listinfo/stoves_lists.bioenergylists.org
>
> for more Biomass Cooking Stoves, News and Information see our web site:
> http://stoves.bioenergylists.org/
>
Thanks
Frank
Frank Shields
Gabilan Laboratory
Keith Day Company, Inc.
1091 Madison Lane
Salinas, CA 93907
(831) 246-0417 cell
(831) 771-0126 office
fShields at keithdaycompany.com
franke at cruzio.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.bioenergylists.org/pipermail/stoves_lists.bioenergylists.org/attachments/20170418/e1fe3858/attachment.html>
More information about the Stoves
mailing list