NWS Central Region Impact Based Warning Experimental Product

[Jeff Duda]

Mike suggests a faster scanning strategy for the 88-D. Tuesday night, KTWX 88-D was running in VCP-212 mode. Let's say the radar could be set up to complete the volume scan in half the time. Would we have been able to spot the vortex and provide enough warning, given it was moving at 75 mph? Hard for me to say.

As is the case with any event (even life in general) there's always room for improvement.

In radar meteorology there really is no perfect setup. If you want to gain one thing, you lose something else in return. If you want increased update speed, you either must sacrifice additional elevation scans or sacrifice accuracy of measurements. Since the 88Ds are meant for surveillance, the radar operator can't just set the radar so that it dedicates all its scanning to one cell of interest. There are usually multiple areas of concern on a radar scan, so there isn't much one can do in those instances. If you scan faster, you'll have fewer pulses per range gate, and less accuracy of your data. Already, the accuracy thresholds are being violated in the fast-scanning VCPs (11, 12, 212 etc.). Plus, with the dual polarization upgrade, the number of pulses is essentially being halved to make room for the vertically polarized pulses, so the accuracy is even worse for radar sites with the upgrade. What good would two-minute updates be if the data were so dirty/uncertain that you couldn't make any reasonable inference from it?

 

[Mike Smith]

If you want increased update speed, you either must sacrifice additional elevation scans or sacrifice accuracy of measurements. Since the 88Ds are meant for surveillance, the radar operator can't just set the radar so that it dedicates all its scanning to one cell of interest.... What good would two-minute updates be if the data were so dirty/uncertain that you couldn't make any reasonable inference from it?

That is exactly what I want, truncate the volume scans in return for more frequent low-level surveillance . In the case of Tuesday night, EAX and ICT provided plenty of high altitude coverage over the northern Flint Hills.

Second, I want to go back to the future. In the 1970's a company called Enterprise Electronics Company (which I believe is now part of WSI) made radars with "sector scan" capability. To illustrate how that worked, let's assume all the storms are in a line about 50 miles away SW through West of the radar. So, the operator sets up the radar to sweep from, say, 210° to 290°. Once every, say, 5-7 minutes, the radar automatically does a 360° sweep to make sure nothing is missed.

It is past time to start thinking outside of the box we created in the 1980's with the current volume scan strategy.

This isn't the only missed event last week. See: www.youtube.com/watch?v=IzwEt02ZVjs&feature=youtu.be While the TV meteorologist is saying there is "no indication" of a tornado an F-2 was causing 4 injuries and destroying 50 homes and damaging about 100 others.

NWS preliminary report here: https://nwschat.weather.gov/p.php?pid=201203040044-KGSP-NOUS42-PNSGSP

Newspaper coverage here: www.charlotteobserver.com/2012/03/04/3065506/storms-rip-across-mecklenburg.html

In this case, the CLT TDWR data (I'm told) did a superior job of depicting the rotation than the WSR-88D's.

We need to make the best use of the best data available. I believe we can do better than sticking to systems that were set up a quarter century ago.

 

[rdale]

More good followup by Jon Davies and "no warning" (well, other than a blanket TOR from a private sector company.)

http://davieswx.blogspot.com/2012/03/harveyville-ks-tornado-article-in.html

 

[Mike Smith]

More good followup by Jon Davies and "no warning" (well, other than a blanket TOR from a private sector company.)

http://davieswx.blogspot.com/2012/03/harveyville-ks-tornado-article-in.html

Since we are the private sector company Jon wrote about (without contacting us or having any first-hand knowledge of our operations or our techniques), the following are facts:

• I don't know what a "blanket TOR" is but our tornado warnings are generally more specific than those of the National Weather Service's
• Our clientele is far more diverse than stated: We have universities, municipalities, and multi-county jurisdictions that sound their sirens based on our warnings.
• Private sector meteorology has a much higher level of accountability than the NWS. If our clients do not like our products or believe they are not getting good value, they cancel their contract and we do not get paid. The NWS collects its money from all of us (via the IRS) regardless of our personal preferences.

Being the author of a book that is highly complimentary of the meteorological community in general, and the NWS in particular, and our efforts to create an effective warning system, I can hardly be considered a "critic" of the NWS. I just wrote an item this afternoon supporting the NWS in its budget problems.

The above are facts. Now, some opinion:

My first concern is always the welfare of the public and society. That requires a strong, vibrant NWS. But, it also requires an NWS that works with its partners to integrate the best science and scientific techniques. I have serious concerns about the direction of their storm warning program in recent months and am voicing them so the public is prepared for not-well-publicized changes in a system that has served them well for so many years.

I'd love to be wrong about the changes in the warning system. I fear I am not.

 

[Greg Stumpf]

I fear something about my position is getting lost in the translation: By representing themselves as able to do tornado warnings tiered by intensity they set themselves up for more "Monday morning quarterbacking," not less. I'm concerned that the April 1 experiment will serve neither the public nor the NWS well.

I've been off ST for quite a while and am now catching up, but this and related comments by Mike can't go unnoticed from me.

I will state up front that I'm not completely wedded to Central Region's impacts-based warning (IBW) demo, but I will say one thing. The warning tiers are NOT based on forecasted intensity. They are based on forecaster confidence. The forecaster has to have confidence that a tornado is occurring before the warning jumps up to the second tier (via credible reports, or a dual-pol debris signature). And the tiered warnings are also based on potential impacts. A forecaster has to have both credible confidence/evidence of a particularly-damaging tornado, and that the tornado is going to have a major impact to population before it jumps to the third tier. The third tier should be very rare, and for the public's sake, let's hope we don't even see one during the entire length of the demo. Ideally, the IBW tiers should score better with each higher tier, because they are based on forecaster confidence, or the probability of the event occurring.

In the case of Harveyville, the forecaster did not have enough confidence that a tornado was occurring or about to occur, and thus did not issue a Tornado Warning. And if this event had occurred during the IBW demo period, it clearly would NOT have been a Tier 2 warning. How could you expect a forecaster to have confidence that a significant tornado was occurring without a report, and without a radar velocity signature? This is regardless of the fact that it was a killer EF-2 tornado that went through the town.

Mike, the Stan Finger article indicates that your company had issued a Tornado Warning on this storm in advance of the tornado. Kudos to you! You guys had a lower confidence threshold for hitting the button based on other indications in the storm. You didn't wait for the velocity signature. You went on a reflectivity signature and storm history. You took the position of "safe over sorry". You got the hit. But at what expense? More false alarms. Like here: http://www.wtop.com/?nid=893&sid=2592581 That's just the nature of the beast when your warnings are issued at a low probabilistic threshold. I suspect so because your clients probably require longer response times to warnings. Thus your clients' probabilstic threshold for taking action should be lower and the cost in doing so can be justified by the potential costs of not taking action.

Wait, did I just insinuate that Mike Smith issues probabilistic warnings?

 

[Mike Smith]

Mike, the Stan Finger article indicates that your company had issued a Tornado Warning on this storm in advance of the tornado. Kudos to you! You guys had a lower confidence threshold for hitting the button based on other indications in the storm. You didn't wait for the velocity signature. You went on a reflectivity signature and storm history. You took the position of "safe over sorry". You got the hit. But at what expense? More false alarms. Like here: http://www.wtop.com/?nid=893&sid=2592581 That's just the nature of the beast when your warnings are issued at a low probabilistic threshold. I suspect so because your clients probably require longer response times to warnings. Thus your clients' probabilstic threshold for taking action should be lower and the cost in doing so can be justified by the potential costs of not taking action.

Wait, did I just insinuate that Mike Smith issues probabilistic warnings?

Hi Greg, thanks for a thoughtful comment!

We have a set of fixed criteria for tornado warnings that our meteorologists use. In both cases, Kansas and Maryland, the thresholds were met so we warned. You are absolutely correct that some of our clients require long lead times so we will warn with a lower threshold in those cases. We adjust our warning strategy to fit client needs rather than a "one fits all" approach, the latter being appropriate to the NWS's work.

That said, it is incorrect that, overall, we have more false alarms. Based on our verifications, we have fewer.

The warning tiers are NOT based on forecasted intensity. They are based on forecaster confidence.

That is not what the Product Description says. Those words are reproduced, verbatim, below. This is clearly based around tornado intensity.

For tornado warnings (TOR), forecasters will be provided three options. The standard or“baseâ€￾ tornado warning represents the first option. The chief improvements for thesebase warnings – likely the most common type issued through the season – will be bulletswhich plainly and clearly communicate hazard and impact information, calls to actionrephrased by social science partners, and tags which identify whether the tornado isobserved or radar-indicated (implied statement of confidence in evidence), predicted hailsize, and the option to add strength of non-tornadic thunderstorm-related wind. Thiswarning type will be selected for cases in which there is credible evidence of a tornado.


The second level of tornado warning – one for which there is substantial evidence of asignificant tornado coincident with a high impact event – will include the phrase "This isa Particularly Dangerous Situation" and incorporate enhanced wording within the secondwarning bullet to identify a high level of risk, dramatic description of expected damageand impacts, and promote serious urgency in taking action to seek shelter immediately.The "PDS" warning will also append an explicit damage threat reference in the form of atag line code "TORNADO DAMAGE THREAT...SIGNIFICANT", rather than simplydiscriminating between observed or probable.


The third and highest level of tornado warning will be reserved for those rare cases inwhich a known, violent tornado is approaching which is likely to experience devastatingdamage - events such as Joplin. For these situations, the enhanced wording will include a"TORNADO EMERGENCY" announcement, the recommended action will be brief,clear and extremely urgent (e.g., IF YOU ARE IN OR NEAR LIBERTY...SEEKSHELTER IMMEDIATELY!), and the tag line will read: TORNADO DAMAGE THREAT...CATASTROPHIC.


Meteorological science does not have the science to do this type of micro scale forecasting. You may have heard that Patrick Marsh's verification of "tornado emergencies" have a "success rate" (i.e., an F-4 or -5 tornado occurring) of 12%!! I don't know of anything that is going to occur in the next three weeks that will magically change this dismal statistic.

With regard to probabilistic warnings, our clients are adamant they want "yes" (a threat exists) or "no" communications.

Thanks again for a thoughtful comment!

Mike

 

[Greg Stumpf]

You are absolutely correct that some of our clients require long lead times so we will warn with a lower threshold in those cases.

By lowering your threshold for those clients, you are issuing probabilstic warnings! Those clients get a low-probability warning. Your other clients get a higher probability warning. They don't really know it, but that is exactly what you are doing.

We adjust our warning strategy to fit client needs rather than a "one fits all" approach, the latter being appropriate to the NWS's work.

Are you saying that the NWS should be issuing one-size-fits-all warnings? Hope not, because that's not what it should be doing! As I've stated many times in my presentations, the public is not a monolith, and they all have varying needs, exposures to hazards, and response times to hazards. Hence, it is innappropriate to issue the same warning for all of them. Just like your clients.

That said, it is incorrect that, overall, we have more false alarms. Based on our verifications, we have fewer.

Any chance we'll ever get to see those numbers? Do you have different sets of numbers for different warning thresholds?

That is not what the Product Description says. Those words are reproduced, verbatim, below. This is clearly based around tornado intensity.

No they don't, and I'll show via my own highlighting...

For tornado warnings (TOR), forecasters will be provided three options. The standard or“base” tornado warning represents the first option. The chief improvements for these base warnings – likely the most common type issued through the season – will be bullets which plainly and clearly communicate hazard and impact information, calls to action rephrased by social science partners, and tags which identify whether the tornado is observed or radar-indicated (implied statement of confidence in evidence), predicted hailsize, and the option to add strength of non-tornadic thunderstorm-related wind. This warning type will be selected for cases in which there is credible evidence of a tornado.


The second level of tornado warning – one for which there is substantial evidence of a significant tornado coincident with a high impact event – will include the phrase "This is a Particularly Dangerous Situation" and incorporate enhanced wording within the second warning bullet to identify a high level of risk, dramatic description of expected damage and impacts, and promote serious urgency in taking action to seek shelter immediately.The "PDS" warning will also append an explicit damage threat reference in the form of a tag line code "TORNADO DAMAGE THREAT...SIGNIFICANT", rather than simply discriminating between observed or probable.


The third and highest level of tornado warning will be reserved for those rare cases in which a known, violent tornado is approaching which is likely to experience devastating damage - events such as Joplin. For these situations, the enhanced wording will include a"TORNADO EMERGENCY" announcement, the recommended action will be brief,clear and extremely urgent (e.g., IF YOU ARE IN OR NEAR LIBERTY...SEEK SHELTER IMMEDIATELY!), and the tag line will read: TORNADO DAMAGE THREAT...CATASTROPHIC.

In each of those cases, I've highlighted the guidance that clearly states that there must be evidence of a tornado to make these distinctions. It is not supposed to be based on a radar velocity estimate. I agree with you that the science is not yet able to do a good job at distinguishing tornado intensity from radar data alone, but there have been some recent studies that are starting to show some success.

BTW - some well-known radar experts (e.g., Burgess) feel that the dual-polarimetric debris signature is pretty fool-proof in identifying tornadoes (but not necessarily intensity), so that is most likely going to be considered substantial evidence of a tornado as well.

With regard to probabilistic warnings, our clients are adamant they want "yes" (a threat exists) or "no" communications.

Again, you issue the warnings, and based on the various probabilistic thresholds appropriate for your clients, you tell them 'yes' or 'no'. I hope you realize that you are providing some clients with low-prob warnings, and others with high-prob warnings. You can't fit one size to all your clients, and the NWS can't fit one size to all the public.

 

[rdale]

Good discussion here... I think Mike's confusion on the probability products is because AW is "filtering out" based on the probability. It's still there...

BTW - some well-known radar experts (e.g., Burgess) feel that the dual-polarimetric debris signature is pretty fool-proof in identifying tornadoes (but not necessarily intensity), so that is most likely going to be considered substantial evidence of a tornado as well.

I think the Roswell - Alpharetta "tornado" from 3/2/12 needs to be researched more before that conclusion gets too heavily circulated. A 'debris ball' was apparent in BR, with high rotation in SRM, and CC making the case too for 30+ miles and nearly 2 hours of warning. All with the "tornado confirmed because of debris ball" wording. Except nothing happened -- not even some wind damage... Here is Mike Gibson's quick guess:

"After looking at the event, or nonevent, in the L2 data, I wonder if the "debris signature" in the Dual Pol data was due to the inflow notch being pinched off. Watch the evolution of BR from 0224 to 0242. You see an open notch that closes off to a BWER. RHO/CC follows the same pattern, a notch that evolves into a "hole". According to the WDTB docs, this RHO/CC "hole" is part of the DP debris signature. It's a particularly weak signal in this case but I can see a user getting confused by it, especially with the strong and coincident signatures in both BR and BV/NROT."

 

[Greg Stumpf]

Here is Mike Gibson's quick guess:

"After looking at the event, or nonevent, in the L2 data, I wonder if the "debris signature" in the Dual Pol data was due to the inflow notch being pinched off. Watch the evolution of BR from 0224 to 0242. You see an open notch that closes off to a BWER. RHO/CC follows the same pattern, a notch that evolves into a "hole". According to the WDTB docs, this RHO/CC "hole" is part of the DP debris signature. It's a particularly weak signal in this case but I can see a user getting confused by it, especially with the strong and coincident signatures in both BR and BV/NROT."

We noticed something similar with a "TDS-like" signature that was NW of KSGF on 2/29, before Branson got hit (different storm though). What started as a CC hole evolved into an alongated "crescent-shaped" minimum along the leading edge of a short convergence zone. But, it was associated with low reflectivities and a lack of a velocity couplet, which are necessary ingredients for a TDS. So we have to be careful when identifying - there will be "mimics", just like you can get a "hook echo" without an actual low-level mesocyclone if the shape of the radar echo "just so happens" to be hook-shaped.

I should also point out that a Tornadic Debris Signature is a dual-polarimetric radar feature, and is not the same thing as a "debris ball", which is a conventional (reflectivity-only) feature. Also, a debris ball isn't simply the ball at the end of a hook. It must be characterized by very high reflectivities that trend upward in intensity in the center of the ball. Also, there should be a trend in upward intensity in adjacent elevation angles aloft as the debris starts getting lofted into the mesocyclone. One neat thing about the DP signature, however, is that after the tornado, you can see the debris signature spread out and loft upwards from the original CC hole with time. It's almost scary what the DP radar can see now - add that to 30-second updates with future rapidly-updating phased-array radar!

 

[Greg Stumpf]

I agree with you that the science is not yet able to do a good job at distinguishing tornado intensity from radar data alone, but there have been some recent studies that are starting to show some success.

I got a PM asking for some more info about this so I'll share with everyone. I'm aware of three studies right now that are trying to relate Doppler velocity signatures to tornado intensity. All three are yet to be published in the formal literature, and only one of the three has been presented at a conference and has something available to peruse online:

http://www.essl.org/ECSS/2011/programme/presentations/21.pdf

The other two studies, and perhaps a follow up to Toth et al., should be presented at the next AMS Severe Local Storms conf. in Nov 2012.

 

[Mike Smith]

In each of those cases, I've highlighted the guidance that clearly states that there must be evidence of a tornado to make these distinctions. It is not supposed to be based on a radar velocity estimate. I agree with you that the science is not yet able to do a good job at distinguishing tornado intensity from radar data alone, but there have been some recent studies that are starting to show some success.

BTW - some well-known radar experts (e.g., Burgess) feel that the dual-polarimetric debris signature is pretty fool-proof in identifying tornadoes (but not necessarily intensity), so that is most likely going to be considered substantial evidence of a tornado as well.


Again, you issue the warnings, and based on the various probabilistic thresholds appropriate for your clients, you tell them 'yes' or 'no'. I hope you realize that you are providing some clients with low-prob warnings, and others with high-prob warnings. You can't fit one size to all your clients, and the NWS can't fit one size to all the public.

Greg, you apparently misunderstood my response at one point. You said the warning categories were based on "forecaster confidence." I don't see those words in anything you have quoted.

Second, please clarify what you mean by it is "inappropriate" for the NWS to have a uniform set of warning criteria. You seem to be saying that it is OK to treat a citizen of NYC differently than a citizen of western Oklahoma. If that is what you mean, I would strongly disagree. The NWS should provide uniform service across the U.S. to everyone.

Of course there is probabilistic element to the warning decision. That is a huge difference from issuing probabilistic warnings. I think issuing probabilistic warnings would be a disastrous step.

Again, great discussion.

 

[Greg Stumpf]

Greg, you apparently misunderstood my response at one point. You said the warning categories were based on "forecaster confidence." I don't see those words in anything you have quoted.

I highlighted them in red - did you not see them? The key word: evidence. In other words, the warnings don't move into the top two tiers without credible evidence, and that means if implemented as is, there should be no false alarms in the top two tiers based on traditional verification.

If you still don't see that or don't believe me, I suggest you contact NWS Central Region reps for verification. They provided us a webinar last week, and clearly indicated that the tiers are based on evidence and not a forecast of tornado intensity. CRH is also going to develop objective verification metrics to ensure that the top two tiers meet the criteria intended. I'm awaiting info from them which will describe this.

BTW, I made the suggestion to them that any upgrade in tiers requires a re-warning, and not an SVS statement. The latter are typically not alerted on NWR (no EAS tone) or for other alerting systems like txt messaging, tv bugs, etc. A re-warning would set off all the bells and whistles again. No decision yet from CRH on this.

Second, please clarify what you mean by it is "inappropriate" for the NWS to have a uniform set of warning criteria. You seem to be saying that it is OK to treat a citizen of NYC differently than a citizen of western Oklahoma. If that is what you mean, I would strongly disagree. The NWS should provide uniform service across the U.S. to everyone.

I'm saying it is not ok to provide the same warning to someone in a reinforced structure versus out in the open (as two examples). The latter requires more lead time than the former.

Of course there is probabilistic element to the warning decision. That is a huge difference from issuing probabilistic warnings. I think issuing probabilistic warnings would be a disastrous step.

Mike, I hope that *someday*, you'll understand what I'm talking about. Your company is already issuing probabilistic warnings, but you are "shielding" that info from your clients. The NWS could do the same with the public that don't understand. Or it could communicate uncertainty in other ways for those members of the public that can and do understand probabilities. Don't think there are any that understand? You might check this out:

http://metofficenews.wordpress.com/2012/01/19/early-results-from-our-record-breaking-weather-game/

Again, great discussion.

Agreed.

 

[rdale]

BTW, I made the suggestion to them that any upgrade in tiers requires a re-warning, and not an SVS statement. The latter are typically not alerted on NWR (no EAS tone) or for other alerting systems like txt messaging, tv bugs, etc. A re-warning would set off all the bells and whistles again. No decision yet from CRH on this.

I am glad they started making some decisions on this and including outsiders with all the time left before testing begins But that would be a HORRIBLE decision not to make. There is no reason at all to have the tiers if they aren't going to reissue a tor on a tier increase.

 

[Mike Smith]

Are you saying that the NWS should be issuing one-size-fits-all warnings? Hope not, because that's not what it should be doing! As I've stated many times in my presentations, the public is not a monolith, and they all have varying needs, exposures to hazards, and response times to hazards. Hence, it is innappropriate to issue the same warning for all of them. Just like your clients.

Since when does the NWS adjust its products for particular clients/industries? That is absolutely not the role of the NWS and is explicitly against their own policies.

 

[Mike Smith]

n other words, the warnings don't move into the top two tiers without credible evidence, and that means if implemented as is, there should be no false alarms in the top two tiers based on traditional verification.

If you still don't see that or don't believe me, I suggest you contact NWS Central Region reps for verification. They provided us a webinar last week, and clearly indicated that the tiers are based on evidence and not a forecast of tornado intensity. CRH is also going to develop objective verification metrics to ensure that the top two tiers meet the criteria intended. I'm awaiting info from them which will describe this.

I spent 45 minutes on the phone yesterday with Ken Harding of Central Region discussing this very topic and I stand by every word I have written on storm track.

Let me see if I can bridge our gap in understanding. The current tornado emergency (TE) is only supposed to be issued in rare, high confidence situations:

“In exceedingly rare situations, when a severe threat to human life and catastrophic damage from a tornado is imminent or ongoing, the forecaster may use the terminology" TORNADO EMERGENCY FOR [GEOGRAPHIC AREA]" in the third bullet of the warning. Additionally, in such a situation,this terminology should only be used when reliable sources confirm a tornado, or there is clear radar evidence of the existence of a damaging tornado such as the observation of debris.â€￾

Yet, per Patrick Marsh's research, the current TE has a dismal success rate of 12%.

The current TE requires "reliable sources" and "clear radar evidence." I fail to see what is going to change between now and April 1 that will cause TE's on that date and thereafter to be more accurate than those of the last 13 years.

Part of the problem is "forecaster confidence," by itself, has little value. Dr. Tom Stewart of SUNYA, who has spent a fair amount of his career researching human factors in meteorology, says, "using more models increases forecaster confidence but it does not increase forecast accuracy." Just because more "evidence" is supposed to be accumulated after April 1, it does not mean the accuracy of the new TE's will be any better than the existing TE's.

My knowledge of the current state of the science is that we do not have the scientific underpinning to skillfully nowcast tornado intensity. If we did, the current TE's would not be so bad. Absent that science, all of the random evidence in the world will have little contribution to the accuracy of a tiered warning scheme.


The insinuations you and Rob are making that I somehow don't "understand" probabilistic warnings are getting a little old. I understand them perfectly. But, they are not germane to what we do. We have built a remarkably successful business by every metric, scientific (accuracy) and business (customer satisfaction, revenue and profit), doing what we do. How many meteorological organizations do you know these days that are both growing and hiring?

We incorporate science and invent and build technology in support of that mission. I am the inventor on 19 U.S. and foreign patents in this area of science and technology.

I'm not saying we are superior, just that our mission is quite different from yours, and thus a very different perspective.

We obviously know quite a bit about communicating the risks of high impact weather. I share my perspective with NOAA and here on storm track because I believe it has value. You are free to disagree.