Erroneous Data

[Jared Orr]

Last night's forecasting workshop got me thinking a lot about erroneous data. We all know "garbage in, garbage out", but I just wanted to get some conversation going:

What are some really practical ways that we can spot or even anticipate inaccurate data? Keeping station density in mind and knowing the common sounding errors (See this thread) are some. But what are some other helpful "sanity checks" that you guys have used when forecasting?

Jared

 

[Brian G]

I found that part of Rich's lecture very enlightening. I also liked how he presented real cases where the data could not be trusted. I definitely learned a lot and it now has me more focused on the topic as well.

I read somewhere that balloons may have a tendency to find and follow thermals. I suppose this would (if true) be more pronounced on days when horizontal convection rolls would be expected. Would this be a source of error and how much if so? I can't remember where I read that so it's possible I'm making it up.

I have seen data deniability studies for the short term models and it appears that aircraft soundings are important in regards to model skill (possibly more so than RAOB launches). To what level should we trust or distrust RAP analysis and forecast soundings? I've always found them useful especially for cities like my home town, St. Louis, which does not have an upper station, but has a busy airport that should be feeding the RAP a steady stream of aircraft derived soundings.

I too would like hear everyone's thoughts on the topic.

 

[MClarkson]

There are a 1000 ways, some obscenely complicated, at observational weighting, interpolation, and bad data checking, in regards to model initialization. The initialization programs that are more complicated (and usually more accurate) often take as long to run as the models themselves. However there are pretty basic checks as well that certainly do not require massive computing.

Is a value physically possible? If you see a report of 320 degrees Celsius, its clearly wrong. Maybe someone added a 0.

Is a value a major departure from what you expect? Become suspicious.

Is a value a major departure from nearby observations? Its probably wrong.

Do all the different platforms agree? Do radar, satellite, and surface obs say the same thing?

Specifically for radar, are echoes in the same place as mountains and are stationary? Then echoes are mountains, not precip.

Then for some of the smaller errors, a little bit of verification stats can go a long way in curing many biases, and then you can use a decision tree to break these down by regime.

 

[Ben Holcomb]

I've found not to trust the AWOS stations at Norman and especially Chandler, OK for dewpoints. Rich touched on that last night.

I look at the mesonet instead, but they may not always be accurate either. I never knew that they modified the soundings on the SPC site to use a virtual temp and that's how they came up with the data below. I will have to use those soundings a little more objectively. Also didn't realize the dip in dewpoint just above the surface was actually a sounding issue. I thought that was just a normal thing.

 

[Jeff Duda]

There are a 1000 ways, some obscenely complicated, at observational weighting, interpolation, and bad data checking, in regards to model initialization. The initialization programs that are more complicated (and usually more accurate) often take as long to run as the models themselves. However there are pretty basic checks as well that certainly do not require massive computing.

Is a value physically possible? If you see a report of 320 degrees Celsius, its clearly wrong. Maybe someone added a 0.

Is a value a major departure from what you expect? Become suspicious.

Is a value a major departure from nearby observations? Its probably wrong.

Do all the different platforms agree? Do radar, satellite, and surface obs say the same thing?

Specifically for radar, are echoes in the same place as mountains and are stationary? Then echoes are mountains, not precip.

Then for some of the smaller errors, a little bit of verification stats can go a long way in curing many biases, and then you can use a decision tree to break these down by regime.

Best reply to this thread so far. For those smaller errors, unless you're part of the data collection, ingest, and/or QC process, chances are that, as an end user of a model forecast or product, you'll just never be able to know for sure if certain data are accurate or representative. Ingestion of satellite products, for example, has been a recent addition to the observations used to initialize NCEP models. While the use of such data has made a big difference over the oceans and sparsely populated areas such as Russia, the differences in, e.g., 500 mb heights over the CONUS are not going to be very big, perhaps not even big enough to register on the synoptic or mesoscales. When doing a broadbrushing analysis, such details can be missed, and those details can turn out to make the difference between a multi-supercell outbreak and no storms. Yet the difference in the products you look at will be essentially imperceptible.

 

[Bob Hartig]

For those smaller errors, unless you're part of the data collection, ingest, and/or QC process, chances are that, as an end user of a model forecast or product, you'll just never be able to know for sure if certain data are accurate or representative. . . . The differences in, e.g., 500 mb heights over the CONUS are not going to be very big, perhaps not even big enough to register on the synoptic or mesoscales. When doing a broadbrushing analysis, such details can be missed, and those details can turn out to make the difference between a multi-supercell outbreak and no storms. Yet the difference in the products you look at will be essentially imperceptible.

Excellent thread. I wondered the same thing last night and emailed a query about how I can learn more about recognizing bad data. Obviously an important part of interpreting a skew-T, for instance, is being able to spot questionable spots in the sounding. But from what you're saying, Jeff, it appears that my options as an "outsider" are going to be limited to common sense, with personal knowledge being the variable.

Rich's and Ben's comments about the unreliability of AWOS were an eye-opener for me.

 

[Jeff Duda]

I've found not to trust the AWOS stations at Norman and especially Chandler, OK for dewpoints. Rich touched on that last night.

I've commented about this on Facebook before (sorry to readers who are not FB friends with me, but I will repost the images here).

Image #1

This first one shows how erroneously high dewpoints measured by the obs managed to get ingested into the SFCOA at SPC and showed up in the mesoanalysis. Nearby obs from the Oklahoma Mesonet (which are generally of the same quality as ASOS observations...quite good) show that the AWOS obs are probably wrong. The AWOS stations indicate 2-m dewpoints in the upper 70s whereas the Mesonet stations indicate 2-m dewpoints are only in the lower 70s.

---------------------------------------------------------------------------------------------------------------------------------------

Image #2

This second image is related to the first. It shows how the ingestion of bad obs impacted other products that depend on 2-m dewpoint in this case. 2-m theta-e and consequently, SBCAPE (really, MLCAPE, too) were all incorrect because those obs made it into the system.

These are examples of the third item on MClarkson's list.