[Stoves] FW: REQUEST for complete sets of raw data of cookstove tests.

[Jetter, James]

To All,

Resending my comment below, even though we've moved on in the conversation, because I've just learned my comment didn't post to the listserv the first time, and I wanted to make sure people know EPA is currently working on a way to share raw data - we'd be very happy to be able to share this information with the community.

Jim

-----Original Message-----
From: Stoves [mailto:stoves-bounces at lists.bioenergylists.org] On Behalf Of stoves-owner at lists.bioenergylists.org
Sent: Sunday, April 21, 2013 5:28 AM
To: Jetter, James
Subject: Request to mailing list Stoves rejected

Your request to the Stoves mailing list

    Posting of your message titled "RE: REQUEST for complete sets of
raw data of cookstove tests.     was Re: [Stoves] Stove testing with
and beyond the WBT"

has been rejected by the list moderator.  The moderator gave the following reason for rejecting your request:

"Please trim the recipient list; it is too long."

Any questions or comments should be directed to the list administrator
at:

    stoves-owner at lists.bioenergylists.org


-----Original Message-----
From: Jetter, James 
Sent: Wednesday, April 10, 2013 12:20 PM
To: 'Paul Anderson'; Discussion of biomass cooking stoves
Cc: Crispin Pemberton-Pigott; 'Ranyee Chiang'; Mathieu Ruillet GERES Cambodia; Dean Still; KARSTEN BECHTEL CREEC; Harold Annegarn; Rajendra Prasad - IIT Delhi; Timothy Roy Longwell; morgan.defoort at colostate.edu; Mitchell, John; Ebersviller, Seth
Subject: RE: REQUEST for complete sets of raw data of cookstove tests. was Re: [Stoves] Stove testing with and beyond the WBT

Dear Paul,

It was good to see you again in Phnom Penh, and I wish we had had more time to talk, but I hope you'll accept my invitation to visit us soon in North Carolina.  I understand you believe stove testing is an extremely important issue, and we completely agree on that point!

You requested raw data to help with ".finding out if results from sub-optimal earlier versions of WBT can be re-processed (processed forward) to be compatible with version 4.2.1."  Our (EPA) testing in 2010 was based on Version 4.1.2.  Changes made in the WBT data calculation spreadsheet between Versions 4.1.2 and 4.2.1 are documented at the web site:
http://community.cleancookstoves.org/files/354

The documented changes made in the WBT calculation sheet between Versions 4.1.2 and 4.2.1 do not make any difference in results from our testing in 2010.  We can discuss each change that was made in the calculation sheet, and I can explain why it doesn't affect our previous results.  Changes made between versions might affect results from some other stove-testing labs.

As Crispin has pointed out, one thing that would make a big difference in results is how "remaining char" is handled in the calculations, but this has not been changed in the WBT protocol or spreadsheet.  Crispin and I have previously discussed this offline as well as on the stoves listserv, and I think Crispin and I agree that if remaining char is discarded in practice, then the calculations in the WBT should reflect that practice.  This is an issue for the next revision of the WBT, and it would especially affect results for charcoal-producing stoves (such as charcoal-producing TLUDs).  Meanwhile, we (EPA) will report future results per the current WBT protocol (energy in remaining char gets full credit in energy calculations), and we will also report results for the discarded-char scenario (energy in remaining char gets no credit).

Results from our 2010 testing showed that a charcoal-producing TLUD stove burning wood-pellet fuel had very high thermal efficiency compared to all other stoves tested.  But if the remaining char is discarded, the efficiency for the TLUD would be significantly lower.  We haven't seen a need to report efficiency with discarded char for that TLUD prototype we tested, because that stove has not been manufactured or disseminated, but for the two TLUD-type stoves included in the ongoing round of testing, we will report results both ways - char discarded and not discarded.

Back in October 2011, Crispin and I spent some good time comparing results from Crispin's proposed (for Version 4) WBT spreadsheet and our (EPA) spreadsheet - I still have the record of our email correspondence.  We generally found agreement between results, and we were able to explain some minor differences in results.  Rather than rehash old data now, I think it's much more productive to continue to work cooperatively together to do the challenging work ahead of us to: (1) refine existing protocols, (2) develop new protocols, as needed, and (3) build up the network of Regional Testing and Knowledge Centers.

I think the Alliance (Ranyee) is doing a fine job facilitating an inclusive open process for moving forward - great discussions and progress in Phnom Penh!

We (at EPA) are working hard on a process (meeting EPA QA and review requirements) that will enable us to effectively share the raw data along with results from our ongoing round of stove testing.  Hope this will be helpful.

Please call me if you would like to discuss this further.

Sincerely,
Jim
__________________________________________
James J. Jetter, P.E.
U.S. Environmental Protection Agency (E305-03) National Risk Management Research Laboratory Air Pollution Prevention and Control Division Research Triangle Park, NC 27711, USA
phone: (919) 541-4830          fax: (919) 541-2157
email: jetter.jim at epa.gov
___________________________________________

From: Paul Anderson [mailto:psanders at ilstu.edu]
Sent: Monday, April 08, 2013 9:18 PM
To: Discussion of biomass cooking stoves
Cc: Crispin Pemberton-Pigott; 'Ranyee Chiang'; Jetter, James; Mathieu Ruillet GERES Cambodia; Dean Still; KARSTEN BECHTEL CREEC; Harold Annegarn; Rajendra Prasad - IIT Delhi; Timothy Roy Longwell; morgan.defoort at colostate.edu
Subject: REQUEST for complete sets of raw data of cookstove tests. was Re: [Stoves] Stove testing with and beyond the WBT

Dear "Guardians of Stove-Test Raw Data", and others with interest in the subject.

1.  I think that I and others have adequately stated the case that we need some (maybe only 10 or 20) data sets of raw data taken during testing of cookstoves with WBT procedures versions 3.0 to the present 4.2.1.  (specific stove names are NOT needed.)   One issue is the prospect of finding out if results from sub-optimal earlier versions of WBT can be re-processed (processed forward) to be compatible with version 4.2.1.

NOTE:    I apologize in advance if this message steps on people's toes.   I assure you that I would not spend my time on this quest for raw data sets if I did not believe that stove testing is an extremely important issue and that the data sets will be useful.   Please take this message seriously.

2.  There are several possible sources:  12 official stove testing centers; PLUS all the owners of stoves who have paid to have their stoves tested and therefore own their results and the raw data.   So the question is:   WILL ANYONE PROVIDE THE DATA SETS?  

3.  PROPOSALS:   (and the specific stoves are not identified. A.  Each testing center provide three (3) complete sets of raw data, or more.
B.  Stove manufacturers provide two (2) complete sets of raw data, or more.
C.  The GACC (that is paying substantial money into testing centers) uses some of its leadership muscle to provide multiple complete sets of raw data.
D.  Somebody individually (or with a small group of supporters) does what needs to be done (and puts everyone else to shame for not being supportive of objective review of the past and current quantitative measurements of cookstoves.
E.  Some other option???

4.  So, who (perhaps all?) with such data sets will fulfill this request?  And receive appreciation from me and others!!

Your replies will be appropriately noted, as will the absence of replies.   Those who offer the data sets will be consulted about where and how the data can be presented for those who could examine the data.   This is not a secret process.

5.  Or will the "Guardians of Stove Test Data" stonewall?   If so, we will ask Why.   And here are some possible answers:

a.  We test stoves but   1) our data is not worthy of being considered, or 2) we only test "for hire" and we will not even contact our clients to see if they will allow the release of their data set(s)., or 3) Other:

b.  We do not consider this issue to be worthy of our time.

c.  We know that the data on (this entity's stove) is faulty.

d.  We have published results about our stoves and we fear that even with re-named files that our stove might be identified and we would look really bad.

e.  GACC told me not to provide the raw data sets, and it gives me funding.

f.  Other responses:

(But such pressure of being named should not be needed, (or not until later messages??)

Dear reader, if you know someone with raw data sets who might not have received this message, please pass this message on to him/her/them/it. 

Paul


Paul S. Anderson, PhD  aka "Dr TLUD"
Email:  psanders at ilstu.edu   Skype: paultlud  Phone: +1-309-452-7072
Website:  www.drtlud.com
_____________________________________________________
On 4/6/2013 10:45 PM, Crispin Pemberton-Pigott wrote:
Dear Paul
 
I think you put that very well.
 
>As far as I know (and I do NOT specialize in this), very few sets of 
>RAW data are available for examination.
 
I think this is correct. We have the spreadsheets - I have many different copies of the UCB/ETHOS WBT going back to 2003. By entering in the output data it is sometimes possible to work out what the numbers were in some cells and therefore make an updated calculation but not always. At a bare minimum we need a screenshot of the Test1-Test3 pages. It is then a matter of manually reproducing the entries.
 
The problem with this approach is that the different versions of WBT 3.0 3.1 and 4.1.2 cannot be differentiated by looking at the Test pages. So it is not known which corrections have already been applied. It is much better to have the spreadsheet if one of those was used.
 
As wood stove results have a high variability it is not possible to use one or two and get a meaningful guide about where it should be on a performance inventory chart.
 
There are, or course, other tests in common use outside the USA. They either treat the raw data differently or they record different things and are not directly compatible or convertible. 
 
Were the calculated performance metrics between versions of tests not so different, based on the math alone, I would be tempted to say use the old ones as a general guide and start with the newest ideas and retain enough information to be able to update them. The situation is however not so favourable as the information provided is most frequently expected to tell the policy maker how much fuel a stove consumes from the environment. That being the case, the divergence is large, sometimes more than 250% of value.
 
With Aprovecho, Berkeley, Colorado State, Univ of Johannesburg, the States of India and China and South Africa, the EU and various US states applying different understandings to the issue, whatever is agreed (which is always an interim solution) must be done with a clear notion of what the questions are and how such question are answered.
 
>Well, having "older test results" possibly deemed NON-applicable would 
>really shake up the stove evaluation situation.   How old is "old"?   Are the EPA Jetter-run tests already "old?"
 
It was agreed in N Carolina this year that the current version (WBT 4.2.1) is valid for general use until updated again. This automatically defines any test performed using WBT 4.1.2 (pre-17 June 2012) or earlier 'old'. The same fuel consumption numbers used as input for previous versions are significantly (meaning more than 1 standard deviation) different, meaning 'detectably different'. That should settle the question of whether correcting forward is needed or not.
 
If we are to still have a 'water boiling test only' as one complete tool, it requires several more corrections and the removal of invalid calculated numbers from the sheet so people stop reporting them as if they tell us something useful. The useful numbers should of course be retained. But prior to that we need to have a conversation about concepts - what the test is supposed to tell us and what to do to answer those questions.
 
Paul, I appreciate the time you are putting into this. It is very important that these issues be discussed so ideas are solicited.
 
Regards
Crispin