Flaws in the EF rating system

[Ethan Lang]

One thing that I have been looking into are, for lack of a better term, flaws in the EF rating system. The main example being the El reno in which was only rated as an EF-3 due to it only affecting rural areas. From my understanding mobile Doppler measured winds over 300 mph. What are your guys' thoughts on this.

 

[Mike Thalman]

I think if it can be proven by mobile radar, they should go with that rating. There are many large tornadoes that hit nothing to determine the current rating system requirements. Likely there would be far more EF5 tornadoes, that are not accounted for which should be.

 

[rdale]

There is a massive effort right now working through all those issues... Meetings have been happening several times a year (which is fast by science standards )

 

[Randy Jennings]

We're talking apples and oranges here. The Enhanced Fujita Scale assesses damage, not wind speeds. The wind speeds associated with the ratings are only proxies for the actual wind speeds. The original Fajita Scale did have a number of limitations including "a lack of damage indicators, no account of construction quality and variability and no definitive correlation between damage and wind speed." (http://www.spc.noaa.gov/faq/tornado/EFScale.pdf). The modifications made for the Enhanced Fujita Scale made great progress in addressed these limitations. I'm sure it can be made better still, but when this topic comes up we are really criticizing the EF scale for not doing something it doesn't claim or even try to do.

Having said that, I think many folks would like for tornados to be ranked based on actual observed wind speeds. This is research that needs to continue, but it too has some serious limitations. While one can always usually assess the damage after the storm, not every storm has a DOW parked next to it. Even when you have a DOW near the storm it is often measuring winds well above the surface. Last year's Elk City OK tornado was sampled by a CSWR DOW at about 600 feet. At what level would a new scale measure winds? What about storms with no DOW near by? If one depends on the WSR-88D network there are areas where the only radar measurements would be over 15,000 feet above ground (i.e. Paris, TX). Another big issue is timing. A lot can happen between radar paints.

I'm a firm believer that DOW or other radar measurements do not belong in the current EF scale. To add them to one storm and not another would pollute the data and make it hard to be used down the road. However, it would be nice if DOW and other radar measurements was better preserved and cataloged along side the EF ratings. While it is easy to find a storms EF rating and archived WSR-88D data, it is hard to find DOW measurements and one would have to do a lot of work just to get measurements on a single storm out of a archived WSR-88D data.

 

[Dan Robinson]

This from Chuck Doswell's blog ( http://cadiiitalk.blogspot.com/2013/06/the-ef-scale-ratings-brouhaha.html )

The EF-Scale was never intended to be a damage scale, but rather is a windspeed scale. The problem always has been that we have so few actual windspeed observations we must use damage to infer the windspeed - otherwise, only a tiny number of events could be rated. That was Fujita's contribution to the science of tornadoes, and it has been useful (albeit controversial) ever since it was introduced in 1971.

The "consistency" argument is traditional within the NWS as a reaction to technological innovation. I could provide numerous examples of the wrong-headedness of this policy, but I'll try to keep this as concise as possible. For instance, should we degrade the data obtained by WSR-88D radars to that from WSR-57s simply to maintain consistency with the older data sets? Should we disregard the dual polarity information of the new upgrades to the WSR-88D radars just to maintain consistency with the old versions of the radar? When something new and exciting comes on line, its capabilities should be embraced by the agency, not rejected as inconsistent with older technology!

 

[Randy Jennings]

Thanks for posting that Dan. Good insight from Chuck's blog. One could of course argue that "intent" and "reality" are different things here. I agree with Chuck that we shouldn't ignore the new data. We just need to make it easy to distinguish from the old data (and new data without the new technology). Perhaps tornados could be classified with two numbers? Say a EF-3-5 where the first number is without the aid of radar and the second it with the aid of radar. Having said that I don't think the second number is useful by itself. Was that a 5 rating at 10m, 600 ft, 15,000 ft? To me it seems like more "database" fields are needed.

 

[Drew Terril]

I do find if fascinating, in a way, how we seem to get hung up on the EF scale and damage indicators vs DOW measured wind speed. I'm curious as to why this is, given our limitations and the fact that we cannot have a DOW on every storm, or even every major storm. Everyone brings up El Reno (why we key on that storm above any other re: rating discrepancies is something that I do not know), and yet I don't hear anyone saying, "If we'd had a DOW on Tuscaloosa or Blanchard or Goldsby in 2011 they might have been EF5s based on measured wind speeds." That aspect of this discussion that comes up time and time again is something I wonder about.

I have to agree with Randy though, simply because we're still not getting ground speed measurements. Everyone likes to cite the DOW measuring El Reno at nearly 300 mph, but at what height was that wind speed measured? If we can get measurements under 10m, it might be a better way to do it, but that would likely require a DOW to put itself into a compromising situation, and as someone with a ton of experience driving vehicles that large or larger, I'm completely against that idea just to get a 10m or less reading. I'm also not sure how the final rating matters as much as we make it out to be, considering they've always been meant to be estimates. Engineers need more hard data to help design things to withstand severe weather events, so I don't see how a rating, in and of itself, without any hard data accompanying it, is useful, weather it's called an EF3 or an EF5. And quite frankly, wind speeds at 200-600m isn't of much value to someone designing a house that might be 10-20m tall at most unless we can find some correlation between winds at that level and 10m and below.

 

[Dan Robinson]

For what it's worth, the consensus among scientists in this field (Doswell, Bluestien, et al) has tended to favor using the DOW/Raxpol data in the ratings. The work that the TWIRL project is doing will hopefully produce some real progress toward reliably correlating DOW-elevation data with what's going on at the surface, moving us toward an acceptance in using mobile radar data in addition to DIs in ratings.

Are there any summaries available online of what is being discussed in the meetings that are happening? It would be fascinating to listen in or even just read some of the highlights of where things are going.

 

[rdale]

Jim Ladue spoke about it at AMS - here's a link https://ams.confex.com/ams/28SLS/webprogram/Paper300684.html

 

[Alex Elmore]

There has been some work done recently by Bryan Smith and others at SPC to look at WSR-88D data and its relation to tornado damage: https://journals.ametsoc.org/doi/abs/10.1175/WAF-D-14-00122.1

My first thought is sure, why shouldn't DOW and other radar measurements be included in the rating process? But I have to agree with what others have said in this thread in that a DOW most likely still isn't going to sample the near-surface wind-field of a tornado. I feel if measured wind speeds are going to be used to determine EF rating, then it needs to be in the lowest 10 meters. One issue that comes to mind with using DOW data is that what if the DOW measures 200+ mph winds a couple hundred meters off the surface, but at the same time as that measurement the tornado caused EF2 damage to a structure. While there are numerous hypothetical situations that support either side of the argument, it's good to think about these things. From what I've read, the TWIRL project may address some of these situations, though it would take a very large number of samples before any potential applications could be applied.

 

[Drew Terril]

There has been some work done recently by Bryan Smith and others at SPC to look at WSR-88D data and its relation to tornado damage: https://journals.ametsoc.org/doi/abs/10.1175/WAF-D-14-00122.1

My first thought is sure, why shouldn't DOW and other radar measurements be included in the rating process? But I have to agree with what others have said in this thread in that a DOW most likely still isn't going to sample the near-surface wind-field of a tornado. I feel if measured wind speeds are going to be used to determine EF rating, then it needs to be in the lowest 10 meters. One issue that comes to mind with using DOW data is that what if the DOW measures 200+ mph winds a couple hundred meters off the surface, but at the same time as that measurement the tornado caused EF2 damage to a structure. While there are numerous hypothetical situations that support either side of the argument, it's good to think about these things. From what I've read, the TWIRL project may address some of these situations, though it would take a very large number of samples before any potential applications could be applied.

This is basically where I am. Now, granted, I don't have any kind of degrees regarding meteorology, but I did spend a nice chunk of my life building things and have a practical understanding on how various weather hazards can affect structures. That said, it's all estimates. What we need from a building standpoint, more than anything, are actual measurements in the lowest 10 meters. Even with DOW measurements 200-600m off the surface, all that we get with the DIs is an estimate of what types of winds can cause that much damage. Key word being estimates. My undestanding is that, among storms that have given us EF5 DIs, we haven't had DOW measurements of those storms since the EF scale was implemented. Please correct me if I'm wrong on this, but the most notable storms that we've gotten DOW measurements from are Moore 1999 and El Reno 2013. To my knowledge, El Reno-Peidmont 2011, Joplin 2011, etc, did not have a DOW nearby, nor was there one near the more notable storms at the top end of EF4 (Tuscaloosa 2011, Chickasha-Blanchard and Goldsby 2011, Pilger, etc.).

The problem is I don't know how we get those measurements with a DOW type setup, simply because of the risk involved. I have a CDL and even severe level straight line winds mean it's time to get off the road, and I don't think the DOW vehicles are much different than some of your heavy commercial vehicles in that regard. Regardless of weight (and I gross close to 80,000 lbs on many of my loads), there's just too much area for the wind to catch. Even an fully loaded big rig, with the weight not stacked more than a couple of feet high in the trailer, will roll with winds in the 50-60kt range.

I wonder if the types of missions the TWISTEX team operated are the only way to get this data, in conjunction with a DOW at a safe distance to get their own measurements concurrently. The designing of the probes wouldn't be terribly difficult, especially since Tim Samaras already had a working model of his own that I can't see any apparent flaws with, but it would take a ton of skill on the deployment side of things. Anyone on the deployment end would have to have a very stringent understanding on the risk management end, not to mention the driving skill to drive a pickup or SUV large enough(and with a proper layout) to expedite the deployment phase in adverse road and weather conditions.

I think we all want some type of correlation between data we're seeing and what's actually going on at ground level. I'm just not sure if there's any other way without some serious coordination between chasers willing to focus on the mission vs "getting the shot" and the science community. Not being a well known chaser or a scientist, there may be something like that going on that I'm not aware of, so forgive me if I'm suggesting something that's already being implemented.

 

[Dan Robinson]

Some interesting info from Tim Marshall on upcoming EF scale changes:

(Start at 1 hour, 10 minutes, 47 seconds) or append this to the URL (our forum software strips the timestamp from the URL):

?t=1h10m47s

 

[Jonathan Beeson]

My biggest knock on the EF Scale is rather how subjective the actual survey process seems to be. I've seen NUMEROUS surveys that basically allude to one rating before applying a technicality to the survey and then using that technicality to justify a lower rating despite in the damage report they basically say "Oh this could have been X rating/It's very plausible this tornado was actually X rating, but because of a technicality we will be rating it Y rating" (I've noticed it most with specifically EF-2/EF-3 and EF-4/EF-5 ratings). Also, I really do believe that downplaying EF-5 level damage due to the belief that an EF-5 tornado is borderline physically impossible/exceedingly rare is a practice, with the most notable instance of that being on 4/27/14 in Central AR.

I really am a strong supporter of an organized, centralized rating team that is assigned to ALL significant tornadoes that report to NSSL or a centralized office. We don't let local WFOs along the coast rate hurricanes, the NHC is the sole office that can do so. Granted, that's a bit of an apples to oranges comparison as we have in the neighborhood of a thousand tornadoes each year and only a handful of tropical systems. However, for significant/particularly damaging tornadoes, I believe that the rating responsibility needs to stay with a centralized team and not each individual WFO. There's too much variability from office to office, and there's also the mess of what happens when a strong/violent tornado crosses CWA borders.

 

[Alex Elmore]

We don't let local WFOs along the coast rate hurricanes, the NHC is the sole office that can do so. Granted, that's a bit of an apples to oranges comparison as we have in the neighborhood of a thousand tornadoes each year and only a handful of tropical systems.

I would say that's an apples to oranges comparison not only based on the number of hurricanes vs. tornadoes, but mostly due to the difference in how they're rated. The Saffir-Simpson scale for hurricanes is based off of the sustained winds of a hurricane, which then relates to an estimated level of damage prior to a hurricane causing damage. The EF scale is based on damage, which then relates to estimated wind speed after the damage has already occurred. While it doesn't rid the system completely of subjectivity, not just any NWS personnel can go out and survey tornado damage. They have to be trained, and they also have a fairly detailed guide of what damage indicators and the threshold of damage relate to an estimated wind speed. I believe that when the damage is in the violent tornado range (EF4-5), usually professionals are called in to assist with rating the damage.

While a centralized team would help to better the consistency of ratings, it would also potentially take longer to survey damage. People usually don't just sit around and wait for their damaged home or property to be inspected before they start cleaning up. On days with lots of tornadoes that need to be surveyed, that could be problematic for low-end damage. Also, if a tornado just snaps a couple trees, it would be much more cost-effective and efficient for the local WFO to handle it then a non-local team.

There's too much variability from office to office, and there's also the mess of what happens when a strong/violent tornado crosses CWA borders.

While I agree that there has to be some variability between offices, it's getting better as time goes on. Most offices are moving towards using the Damage Assessment Toolkit to perform damage surveys, which is definitely a step towards better consistency:
https://ams.confex.com/ams/97Annual/webprogram/Paper312451.html

Regarding tornadoes crossing CWA boundaries, how the surveying is handled varies case-to-case. Sometimes WFOs will team up to survey a track together, sometimes one will take responsibility of the entire track, or sometimes they'll survey their own portion of the track. Just because a tornado is labeled as EF5, doesn't mean it caused EF5 damage for it's entire track. Usually damage in the range of EF4-5 is relatively sporadic and short-lived. If a tornado caused a higher rating of damage in one CWA than it did before/after it crossed into another CWA, it's not necessarily a sign of inconsistency between WFOs, but just the degree of damage caused by the tornado in their respective CWA.

 

[rebelrage]

I think the EF scale has its place the way it is but what if we had to rating systems the EF scale and something new that included measurements