April 27, 2011 outbreak - NWS service assessment

[Mike Smith]

Patrick,

You can read Warnings, which was finished in late 2009, and I state that false alarms were the next big thing we need to address.

When I started in operational meteorology, the PoD was around 18%. Given that perspective, I believe younger meteorologists may not fully appreciate how far we have come.

One of the things that would help is candid post-mortems of all major events like the one Mike Umschied has up on his blog right now: www.underthemeso.com/img_highplainswx/7/Fig-18.jpg I salute Mike for doing it!

I'm going to do mine tomorrow morning since it is still snowing in eastern Kansas.

I have long thought that we are not candid enough with ourselves in meteorology. Using "best practices" and getting honest feedback after every big event would improve the FAR without any new science based on my almost 40 years (sigh!) of observing forecasters in action. My team didn't/doesn't always like getting the feedback but it clearly makes them better forecasters.

Mike

 

[mwenzbauer]

On April 27, 2011, a series of devastating tornadoes struck the southeastern United States.
This tornado event was the third deadliest in the country since systematic tornado record keeping
began in 1950. With 316 fatalities (31 in Mississippi, 234 in Alabama, 32 in Tennessee, 15 in
Georgia, and 4 in Virginia), it follows only the 1974 Super Tornado Outbreak (368 deaths) and
the 1965 Palm Sunday Tornado Outbreak (337 deaths).

Just curious where these fatality numbers come from? These numbers for Palm Sunday '65 and the '74 Super Outbreak seem higher than I've seen reported elsewhere... Wikipedia shows 271 for Palm Sunday, while TornadoHistoryProject shows 260... nowhere near the 337 reported here... the Super Outbreak shows 319 on Wikipedia and 310 on TornadoHistoryProject (April 3 and 4 combined)... closer, but still not really that close to what was reported here...

 

[rdale]

One of the things that would help is candid post-mortems of all major events like the one Mike Umschied has up on his blog right now

It's always good to do a review, but I don't think that would have much impact on the FAR as a whole. Changing the way we verify warnings, and allowing for different "types" of tornado warnings (separating landspout warnings from EF5 wedges) need to be incorporated first. The current system offers no incentive to reduce FAR.

 

[Elaine Spencer]

Another problem with FAR may be that the general public could have a different idea than meteorologists have of what constitutes a "false alarm." They may think, for example, than even if a tornado actually is sighted and causes damage in a warned area, if it does not hit THEIR neighborhood or community, for them it is a "false alarm."

Some of you have mentioned responses to the Joplin assessment, and others, in which residents said they didn't heed warnings because "storms always miss us" or something to that effect. How much of a "miss" are we talking about here? 5 miles? 10 miles? Does "false alarm" mean no tornado damage or confirmed sighting at all, or tornado damage a certain distance outside the warned area, or what?

 

[Mike Smith]

Power Failures?

Another problem with FAR may be that the general public could have a different idea than meteorologists have of what constitutes a "false alarm." They may think, for example, than even if a tornado actually is sighted and causes damage in a warned area, if it does not hit THEIR neighborhood or community, for them it is a "false alarm."

Some of you have mentioned responses to the Joplin assessment, and others, in which residents said they didn't heed warnings because "storms always miss us" or something to that effect. How much of a "miss" are we talking about here? 5 miles? 10 miles? Does "false alarm" mean no tornado damage or confirmed sighting at all, or tornado damage a certain distance outside the warned area, or what?

While I doubt any resident of Joplin believes the second warning and siren activation on May 22 was a false alarm, we need some metric for non-catastrophic situations.

I gave this subject some thought when I was writing the hurricane chapters of Warnings and came up with this: Did the power fail?

Whether it is a tornado, hurricane, or derecho, the power fails in a geographically larger area than the area directly damaged by the storm (i.e., your home is intact but may be without power because of a downed transmission line 2 miles away). I like it as a metric because utilities can tell us after the fact the numbers and locations of the power failures.

So, the distribution of power failures would seem to be correlated to the "extent" of the storm and whether someone would perceive a given warning as a false alarm.

 

[Zach Young]

So, the distribution of power failures would seem to be correlated to the "extent" of the storm and whether someone would perceive a given warning as a false alarm.

Do you mean to say that if the power does NOT go out in a given area, the event is more likely to be perceived as a false alarm? A problem I see with this is that there a large number of power failures that are simply caused by lightning. Some of those failures are sometimes quite widespread if a critical component of the transmission system is struck. I do agree that if the power goes out due to a tornado, it could "bring it home" for otherwise unaffected people in the warned area.

 

[Mike Johnston]

I disagree. I don't think you're going to see any significant reduction in FAR from the NWS over the next several years. I've said this before and I'll say it again: even though I'm not with the NWS so I don't know this for sure, I'm pretty sure the NWS aims for high POD almost regardless of the FAR that comes with it. This is because research has shown that people consider being killed by severe weather without warning to be one of the worst things possible.

That said, I think improving how the gravity of the situation is conveyed by improving the text part of a warning (assuming no other major changes to the warning system occur) is at least as important as reducing FAR in warnings, and likely more important.

Jeff, my comment wasn't conveying how things actually are as much as how I would hope they will be.

I agree that POD is and should be the supreme stat as far as warnings are concerned. However, I think bringing down the FAR is the next frontier in making the overall warning system more effective, in that in the long term it will bring more credibility to warnings.

I'm hardly an insider to NWS operations, but I wonder if the NWS does any statistical sampling of false alarms. I would guess there is some mix of 1) mistakes in interpretation of radar information, 2) shortfalls in understanding of the atmospheric environment, 3) just a general conservative (meaning CYA) bias, and 4) other factors. Perhaps some common errors contributing to false alarms could be identified and incorporated into training.

As far as communication of warnings, I'm sure that wording could be tweaked here and there for the better, but the real progress will be made through improvements in forecast skill which includes reducing the FAR.

 

[Jeff Duda]

3) just a general conservative (meaning CYA) bias

I think that's the biggest contribution.

 

[Juston Drake]

When it comes to fixing the FAR issues, you must go to the root of how a storm becomes tornado warned or not. With the way our current system works, the NWS issues a tornado warning when a storm has a sufficient amount of rotation being detected on radar or if a tornado has already been observed with the storm. The rotation that the radar is detecting, in almost all cases, is from the mesocyclone, not an actual tornado. There in lies our problem. The tornadic circulation itself is on such a small scale that only the mobile dopplars that are allowed to get right next to the updraft are able to pick up on it. Since the NWS is issuing warnings based on the amount of rotation inside the mesocyclone of the supercell, it is going to be difficult to significantly reduce the FAR because as we know there are hundreds of supercells that form a year that never end up producing a tornado. Because the radar can only take into account the magnitude of the rotation in the mesocyclone, and it can’t detect if the surface conditions are even conducive for a tornado to form (the supercell might be elevated) along with other small scale factors such as storm dynamics which can increase or decrease a storm’s tornado possibilities, there will also be warnings issued for supercells that cannot or would not produce.

Another factor that also complicates the FAR issue that doesn’t even have to do with the warnings is just a lack of education out there in the public. You have no idea how many people out there don’t know the difference between a tornado watch and a tornado warning. There are people out there who think a tornado watch is the same thing as a warning. They will bring up we have been under a tornado warning a hundred times and this and that, when in reality most of those times their area was just under a tornado watch. This further gives them the impression that our tornado warnings are not accurate, when in fact it wasn’t a warning at all but a watch. This is where I think the local new stations can help out a lot reiterating each time, if that is required, the difference between a tornado watch and a warning whenever either one is issued. This will help out the forgetful people who tend to not retain information as well.

While I’m on the subject of the media, I personally think there needs to be an improvement of getting the information out to people regarding watches and warnings on the radio and TV. I can’t tell you the countless times we have been out chasing, and we couldn’t find one of the radio stations that were reporting anything about the tornado warned storms that were out there. We like to tune in to the stations to hear if anything had been reported on any of the cells we were not on, but we couldn’t find a single station talking about any of the storms. Any commuter out there who isn’t paying attention to the weather wouldn’t of had any idea of how serious the situation was near any of the storms. Because of the expansion of cable over the years, more and more people are not watching the local channels anymore. These channels are the only ones that show the different watches and warnings in a person’s particular area. I think it is time to expand that kind of graphic (the colored coded map) to the other cable channels, so that we will know ANYBODY who is watching tv is at least aware of what is going on with the weather.

 

[Todd Renshaw]

The most concise, perfectly-worded warning may not make a difference to the public's perceived FAR. It all comes down to if/when the EMs set off the sirens and what the media says. The siren situation will, hopefully, continue to improve. (And although I think most tv forecasters do an admirable job, there's always the local station's pressure to sensationalize minor events into something apocalyptic. Gotta keep those viewers watching. And that doesn't help the perceived FAR.)

My (semi-informed?) opinion is that when a siren goes off and there's no damage*, not even a downed tree limb, within an individual's neighborhood, that's perceived as a false alarm.

*I'd agree with Mike that a power outage counts. (Legitimizes the siren going off.)

Anyway, going back to the point of Jeff's original post about the language used in the warnings ... I think you can use all the scary sounding, plain english, ordinary Joe language you want, but the public at large isn't even reading those, so why bother. Do what's the most concise and understandable for the real audience; the WFOs, the media and the weather-heads like us. Maybe part of the problem is that these warnings take on the format of a form letter. (Certain key language is used again and again. If situation "A" happens, language "X" is inserted.) Throw in a rarely-seen bit of text that's outside the norm and many people will miss it or misinterpret it. Again, only a semi-informed, layman's opinion. You experts can go ahead and pounce now.

 

[Jeff Duda]

Anyway, going back to the point of Jeff's original post about the language used in the warnings ... I think you can use all the scary sounding, plain english, ordinary Joe language you want, but the public at large isn't even reading those, so why bother. Do what's the most concise and understandable for the real audience; the WFOs, the media and the weather-heads like us. Maybe part of the problem is that these warnings take on the format of a form letter. (Certain key language is used again and again. If situation "A" happens, language "X" is inserted.) Throw in a rarely-seen bit of text that's outside the norm and many people will miss it or misinterpret it. Again, only a semi-informed, layman's opinion. You experts can go ahead and pounce now.

You make a good point with that argument. Joe Q. Public probably doesn't read past the "XXX warning for XXX County" part of most warning texts that scroll across the screen. However, if that is true, then already tornado emergencies are completely useless since the mention of "tornado emergency" comes so far down in the warning text. And the fact that some broadcast meteorologists probably hype up an otherwise not-so-threatening event doesn't help their case when there is a life-threatening event because their message will sound the same in those cases. Then Joe Q. Public won't know the difference and he/she will not act differently. That's tough to fix since each broadcaster does it differently and there is no written rule on how broadcast meteorologists are supposed to read warnings.

 

[Todd Renshaw]

And I don't mean to imply, Jeff, that your questions about clarifying/updating the warning language are useless, just that it should be improved in a way that best benefits the intended audience. If the audience is having trouble sorting through the clutter in order to quickly assess the threat level, then that is a problem. That audience must then disseminate the information in a way the public can understand. But I have no expertise in what the WFOs, EMs and media would want in order to make that easier.

 

[Greg Stumpf]

Because the radar can only take into account the magnitude of the rotation in the mesocyclone, and it can’t detect if the surface conditions are even conducive for a tornado to form (the supercell might be elevated) along with other small scale factors such as storm dynamics which can increase or decrease a storm’s tornado possibilities, there will also be warnings issued for supercells that cannot or would not produce.

Best practices in warning decision making training involve integrating data from other sources besides radar, especially knowledge of the near-storm environment (NSE). If there is sufficient confidence that the supercells are elevated, then a strong mesocyclone on radar should not be enough to flip the switch on a TOR.

As for the "false alarm perception" issue, there are a number of hypotheses floating around right now, all still requiring careful social science research. These include:

1) The areas of verified warnings not affected by the tornado (still counted as a "hit").

2) Warning systems still triggered for whole counties (weather radio, terrestrial radio, television crawls, some siren systems).

3) Warning systems triggered for non-tornadic situations

4) Warning systems triggered for adjacent counties (e.g., sirens, non-SAME weather radios).

5) Television interrupting all viewers in DMA for small threat areas

Regarding #1, this is an interest area of mine, as I am developing a geospatial warning verification system that takes into account, among other things, false alarm area and false alarm time even for "verified" warnings. One aspect of my system is the variable "buffer zone" I can give around a tornado to determine how close one needs to be in order to need to be certifiably warned. I'm hoping that social science can help determine what that distance should be, "How close should a tornado be to you to require a warning?"

Regarding #2, one of the major recommendations from last week's Weather Ready Nation workshop is to add the polygon coordinates to the NWR tone burst. Midland Radio has a 5-year old patent for a "locatable" weather radio (GPS or manual lat/lon entry) ready to exploit this information, and I can't wait.

 

[rdale]

So they would be the sole source vendor? Hmmm...

 

[Greg Stumpf]

So they would be the sole source vendor? Hmmm...

Midland? Dunno. All I know is what the Midland rep said about the patent at the workshop.