[Stoves] Irrelevant lab testing - for what purpose?

Sun Jun 4 12:49:47 CDT 2017

Dear Nikhil

  1.  How can you tell "the uncertainty of the Aprovecho-made PM test equipment is at least 50."?

I referred specifically to the PM2.5 readings. The hardware was produced with tea dive of some experts in PM measurement and one of them told me that the numbers were probably good to +-50%.

The PM number is calculated from a combination of measurements and it is a mini-version of the EPA dilution tunnel system with much lower accuracy. The detector was (originally) described as ‘a smoke detector running at 100 times its design limit’.

The PM number considers the velocity of the gases in the dilution tunnel, the ambient pressure in the tunnel and lab, the temperature at the tunnel exit, the manner in which the sample is collected (isokinetic or not) and the counter itself. All the measurements have uncertainties. Propagating them through the calculations provides an overall uncertainty. But there are other considerations. The method applied is a carbon balance method which is not appropriate for solid fuels burned inhomogeneously. It gives the wrong answer based on the inhomogeneity of the combustion of carbon. The method does not consider the water vapour in the gas stream which can vary a lot (it is unmeasured) and it has no check on the oxygen content to confirm the carbon measurement values. Identical stoves in terms of actual performance on PM emission metric will rate very differently if one is a stick burner and the other is a char-making TLUD.

Considering the equipment, the method, the lack of calibration, the uncertainties, +-50% is not an unreasonable estimate. The thing is, if you dig into these things, the absolute numbers reported also have an impact on the uncertainty value. If you test a very clean small stove in the EPA chamber at Raleigh, the scatter is the results will be much higher than for a ‘dirty’ stove-fuel combination. The reason is that the limit of determination (LOD) becomes a problem. The accuracy of a PM detector for a very clean sample of very small particles is not great. A Dusttrak, for example, counts 20 PM0.1 particles if it sees one. A GRIMM 11-R counts 3 if it sees one PM0.22.  Wood combustion makes small particles almost exclusively. This the correct uncertainty rating of the LEMS/PEMS system – or any other - is a formula, not a single number. Below some value of emittant, the system is not able to make any defensible claim at all (i.e. the measured value is less than 3 times the LOD). When using high dilution systems like the EPA tunnel this limit is reach far sooner than with, for example, the SeTAR variable dilution system (no comparison number available) or the CONDAR from Canada (shown by Brookhaven Nat Lab to be about 5 times ‘better’  for a clean stove).

  1.  You say, "Unless the EPA used the same defective calculations.." Is this not a rebuttable presumption -- that EPA did use the same calculations and perhaps the same equipment?
Th EPA initially reported their WBT results as if they have been done using Aprovecho’s equipment and method, but in reality behind the scenes there were far more calculations going on because the equipment is not the same. At all. It is obvious that to get a number from a stove the appropriate calculations are made based on the test apparatus employed. When I asked Jim Jeter for a copy of a spreadsheet for a particular test, the response was that they used their own method to get the result. The question arises as to whether the method they use includes all the same errors incorporated in the WBT spreadsheet. Different versions have different numbers of systematic errors. When the PM emissions were added and following the IWA, additional metrics were incorporated (3/9 were not valid to begin with) the number of systematic errors leap. There were about 120 for a while. Now there are about 75. However there are two ‘versions’ of the WBT: one for the LEMS/PEMS system and another for the public to use. They give different answers. Even if you use your own system, you have to choose which set of errors you want to incorporate: the LEMS version written by Nordica (which is slightly better) or the GACC version that you can download from the GACC website.
I have no idea what is incorporated in the EPA version. Do they first calculate the real emissions then put numbers into the WBT spreadsheet to get the macerated results? No one is telling. It is a mess. The Stove Comparison chart has the results of many stoves tests that were made using different versions of the WBT spreadsheet, which all give different answers!  There is no discussion of how these numbers were cooked up in the document. It does bother to say that coal stoves, which can burn far cleaner than any wood stove seen to date, only ‘appear to be clean’.
Yeah, well, appearances can be deceiving.
Did the EPA ever do side-by-side comparisons of the PEMS Hood and their system? No idea. I hope so. The LEMS system can give an indication of performance relative to another burn. Absolute numbers, meaning certification, no. It would have to be calibrated daily to do that and the whole system upgraded. That is why the Round Robin results are so poor.

  1.  If it didn't, how did ARC come around to promising certificates of ISO Tiers?
Well it may be slightly overstating its case. The only reviews I know of for the equipment are my own (unpublished) and one by Berkeley (currently on hold). No one knows what the ISO tiers are, if there even are some, and they would have to first be checked for validity (following the fiasco of the IWA producing invalid metrics this is a ‘given’).  ARC may be using a test system we know nothing about, not the LEMS. If it is the LEMS, they will have to produce some form of certification or public claims showing they can meet the standards necessary to certify anything.
I know that the SEET Lab in Ulaanbaatar is ISO 17025. It uses the SeTAR Dilution system, not the EPA-style high velocity dilution tunnel.  At the moment they are only allowed to issue ‘certified’ results for one particular test sequence. To be able to perform other test sequences and certify them they have to be validated against another lab using the same system, presumably CAU or the new Energy Ministry lab in Jakarta (which is really nice).
The test method incorporated in ISO 19867 Part 1 is not the WBT and it has not been externally reviewed yet. Its novel test method may not pass the sniff test.  It also may changed between now and publication as a Draft International Standard. As far as I know the DIS will not contain any uncertainty interval requirement nor any method for how to place stoves on tiers based on the uncertainty of the results.  The EU has a method and applies it rigorously.
Regards
Crispin

-------------- next part --------------
A non-text attachment was scrubbed...
Name: winmail.dat
Type: application/ms-tnef
Size: 53028 bytes
Desc: not available
URL: <http://lists.bioenergylists.org/pipermail/stoves_lists.bioenergylists.org/attachments/20170604/61816462/attachment.bin>