Coming up with an objective way to rank chase days/seasons

[Dan Robinson]

(Updated with latest metrics and definitions, Version 6, December 2019). The QI or Quality Index is a proposed method to quantify the overall quality of a storm chasing event using objective criteria. The term "quality" itself is subjective, but we will define and use the following factors to determine the final QI: daytime photogenic tornadoes, storm structure, lightning, terrain, road network, forecast lead time, storm impacts, rare features and human impacts.
The following is a screen capture of the most recent application (v6) of the QI-S formula to a selection of chase days. QI-S of 10 and up is "good", 20 and up is "exceptional":



The "QI-S Base" is the calculated value before considering negative human impact metrics (what the QI-S would have been if the tornadoes had not hit anything).

QI: Definitions & metrics

• Standard chaser positions: The most common storm-viewing positions employed by chasers, mainly from south, east and north of a mesocyclone. Distance from the storm is not a factor considered by the QI.
• Primary targets (PT): Traditionally-evident targets where the majority of chasers ended up. PTs include but are not limited to dryline bulges, warm fronts, triple points, outflow boundaries, boundary intersections, upper-level speed maxima, etc. There can be multiple primary targets in a single event (common, for example, during big outbreaks).
• Secondary targets (ST): Non-traditional target areas such as cold-core, DVCZ, upslope or Midwest warm/stationary front storms where only a small percentage of chasers deliberately target. Note that these types of targets can become primary if the traditional target areas are too marginal or not present. Secondary targets that become primary should be counted as PTs in QI calculations.
• Daytime photogenic tornado (DPT): The number of daytime tornadoes with complete structure consisting of a funnel and/or debris spanning at least 75% of the distance from ground to cloud base, clearly visible to observers at least 180 degrees surrounding the tornado (standard chaser positions). Photogenic tornadoes assume good visibility to all chasers in the vicinity (distance from the tornado is not considered by the QI).

ABOVE: Examples of daytime photogenic tornadoes. Low contrast does not disqualify a tornado from being considered "photogenic" if the low contrast is due to viewing from great distances and not from heavy precipitation surrounding the tornado.

• Standard tornadoes (T): The total number of tornadoes visible from standard chaser positions during a chase in one target area - including photogenic tornadoes, tornadoes at night, or low-contrast rain-wrapped tornadoes (as long as at least one edge of the condensation funnel and/or debris cloud can be discerned visually and on photos and videos).
• HP tornadoes (HT): The number of tornadoes only visible from within the notch of a high-precipitation supercell. (Tornadoes counted as HTs should not be counted in the T total)
• Good structure (S): Well-developed and symmetric supercell structure, including high-contrast striations, banding and inflow tails.
• Good lightning (L): Frequent, dramatic, remarkable and/or otherwise photogenic lightning accompanying the storm for at least 30 minutes.
• Terrain factor(TF):
  • Good: Flat terrain with few trees or hills (example: western Kansas)
  • Fair: Mostly-flat terrain with a few hills and viewing obstacles (example: eastern Kansas)
  • Poor: Terrain with mostly trees and/or hills, few clear views (example: eastern Oklahoma)
• Roads factor(RF):
  • Good: 1 or 2-mile grid, average spacing between paved or gravel secondaries no more than 4 miles
  • Fair: No grid with half or more of secondaries paved or gravel. Average distances between paved/gravel roads between 4 and 6 miles.
  • Poor: Few secondary roads, most secondaries dirt. No grid with average spacing between paved/gravel routes of more than 7 miles.
• Forecast lead time(FL): Forecast lead time refers to the ability of chasers to see the event coming in enough time to make a decision to travel to the target. "Tornado potential" refers to at least a 5% SPC tornado risk or equivalent level of confidence.
  • Evening before & prior: Tornado potential evident before and up to 00z model runs the previous day
  • Morning-of: Tornado potential only evident in morning data (day of the event).
  • None: No clear tornado/supercell potential evident (mesoscale accident).
• Rare features(R):
  • Multiple simultaneous tornadoes
  • Sunlit tornadoes
  • Audible roar
  • High-contrast mammatus covering at least 1/4 of the sky
  • Prolific upward lightning from towers and/or wind turbines
• Historic (H): Tornadoes/supercells that broke records or were otherwise exceptionally rare in their strength, number or configuration. NOTE: tornadoes that caused major disasters should not be counted as historic at the same time (the "major disaster" metric covers this aspect of historical significance).
• Negatives(N):
  • Problematic traffic (TRF): Traffic (from chasers, locals or both) interfered with more than 10% of the chase in terms of slow travel speeds, delays at intersections and most available pull-offs occupied.
  • Damage (D): Tornadoes that produced significant damage to homes.
  • Human impact: tornadoes that caused injuries (I) OR fatalities (FT).
  • Major disaster (MD): tornadoes that caused destruction to more than 1/3 of a town, more than 30 homes and/or 10 or more deaths.

Target-Level Quality Index (QI-S)

The Target-Level Quality Index or QI-S considers the quality of a single chase target, either primary or secondary. The QI-S evaluates the best possible outcome of a reasonable chase involving one or more storms within reach of that target. The QI-S can cover multiple storms if all of them were observed by the majority of chasers in the target (example: May 5, 2007 involved tornadoes on 3 different storms in one central Kansas target).
The proposed formula (current v6) is as follows:

QI-S = ((DPT*300)+(FL*100)+(H*500)+(TF*50)+(RF*100)+(R*100)+(TRF*-100)+(D*-150)+(I*-350)+(FT*-500)+(MD*-1200)+(L*100)+(T*150)+(HT*70)+(DPTmins*1.1)*(60-speed)+(Smins/30)*(60-speed))/100

Event-Level Quality Index (QI-E)

The Event-Level Quality Index or QI-E is the average of the QI-S for all primary targets. If secondary targets are present, the QI-E is the weighted mean of the primary targets average QI-S (weighted at 75%) and the secondary targets average QI-S (weighted at 25%).

Seasonal or Monthly Quality Index (QI-Y or QI-M)

Seasonal or yearly QIs are simply calculated as the sum of QI-E for all dates in the period. The larger the QI-Y or QI-M, the better the season, month or other defined period.

Javascript Calculator

Online QI-S calculator is available here:

QI-S Calculator

Spreadsheet formula

You can download an OpenOffice Calc spreadsheet file containing the formula here:

quality-storms-v6.ods

The formula in spreadsheet form is as follows:

OpenOffice Calc:
=ROUND((((C2*300)+(J2*100)+(K2*100)+(L2*100)+(M2*100)+(N2*100)+(O2*100)+(W2*100)+(X2*100)+(U2*100)+(V2*500)+(P2*-100)+(Q2*-150)+(R2*-350)+(S2*-500)+(T2*-1200)+(D2*100)+(H2*150)+(I2*5)+(E2*1.1)*(60-G2)+(F2/30)*(60-G2))/100))

Excel:
=ROUND((((C2*300)+(J2*100)+(K2*100)+(L2*100)+(M2*100)+(N2*100)+(O2*100)+(W2*100)+(X2*100)+(U2*100)+(V2*500)+(P2*-100)+(Q2*-150)+(R2*-350)+(S2*-500)+(T2*-1200)+(D2*100)+(H2*150)+(I2*5)+(E2*1.1)*(60-G2)+(F2/30)*(60-G2))/100), 0)

 

[John Moore]

I'm not sure how to do it in a formula, but I think 5/25/16 should have come out higher. I was at the 5/24 and the 5/25 event, and many of us thought the 5/25 was a best of career storm, or close to it. 5/24 was amazing in the quantity of TOR's and the number of simultaneous ones, but 5/25 had stunning structure, was very photogenic with good lighting, and ambled along leisurely for a very long time.

 

[Dan Robinson]

I'm not sure how to do it in a formula, but I think 5/25/16 should have come out higher. I was at the 5/24 and the 5/25 event, and many of us thought the 5/25 was a best of career storm, or close to it. 5/24 was amazing in the quantity of TOR's and the number of simultaneous ones, but 5/25 had stunning structure, was very photogenic with good lighting, and ambled along leisurely for a very long time.

The only reason that date didn't rank higher is because there were two targets that day, and only one of them produced a photogenic tornado before dark. This formula considers all primary chase targets of a day together instead of just one storm. The other target attracted close to half of chasers, and since it busted, brought down the score for that day.

A storm-level ranking that takes into account total photogenic tornadoes, total time that photogenic tornadoes were in progress and storm speed would probably place 5/25/16 in the top 5.

The formula for single storms using the same variables should be:

=ROUNDUP((((D2*150)+(G2)+(F2*15)+(H2)*(60-J2)+(I2/30)*(60-J2)))/100)

 

[Michael Snyder]

I'm not sure how to do it in a formula, but I think 5/25/16 should have come out higher. I was at the 5/24 and the 5/25 event, and many of us thought the 5/25 was a best of career storm, or close to it. 5/24 was amazing in the quantity of TOR's and the number of simultaneous ones, but 5/25 had stunning structure, was very photogenic with good lighting, and ambled along leisurely for a very long time.

5/25 was nice, but I'll disagree massively on it being the better day vs Dodge City, the hand-offs, the 3+ photogenic tornadoes, the multiple tornadoes down at the same time.

 

[Dan Robinson]

Here is the storm-level formula applied (we'll call it QI-S) - 5/25/16 places 3rd with this method:



Again, 3 and up is good, 10 and up is exceptional.

Spreadsheet formula is as follows for anyone who wants to play around with it:

=ROUNDUP((((C2*150)+(E2)+(D2*15)+(F2)*(60-H2)+(G2/30)*(60-H2)))/100)

Total number of tornadoes overall (photogenic and non-photogenic) is probably a factor that should be included (that would raise Leoti's score for example) but I'll tackle that some other time.

 

[Jeff Duda]

This is a good idea and it appears to be unmasking some good results, but just keep in mind that there is still substantial subjectivity in it, especially with what is regarded as a "photogenic" tornado or storm structure. Also, are you using the average forward speed of only the photogenic storms, of all storms in the primary target area(s), of only tornadic storms whether they were photogenic or not, or something else?

As long as your classifications can be defended well, I say this is as good a measure of a chase quality as any. But as with everything in life, your mileage may vary - some people will have career days with events rated QI 5 and under, whereas many people (like me) will find ways to screw up a QI 25+ type event and therefore won't think much of the day in retrospect.

Also, someone is going to have to maintain this database. I take it you will handle that, Dan.

 

[JF Massicotte]

Chasing will always be somewhat subjective. I like how he tries to account for the success likelihood into his index. However, one improvement that could be made is add a measure of how predictable the PT is. For example, Bennington had one primary target, but the forecast confidence wasn't too high, so a chaser living in OKC will likely not take a PTO for that day. That's different for chaser on a chasecation with nothing else to do than to try that one PT.

Adding a factor for the primary target predictability or confidence could adjust the scale a little bit. I know it would just add subjectivity, but I think it is relevant in our case here.

 

[Dan Robinson]

Here are some ideas for definitions and refining the index, along with suggested multipliers and factor values. These multipliers and factors will take some finessing to make this work, these are just preliminary.

Primary targets: traditionally evident targets where the majority of chasers ended up. Including but not limited to dryline bulges, warm fronts, triple points, outflow boundaries, boundary intersections etc. There can be multiple primary targets in a single setup (common during big outbreaks)

Secondary target: non-traditional target areas such as cold-core, DVCZ, upslope or Midwest warm/stationary front storms where only a few chasers deliberately target. Note that these types of targets can become primary if the traditional target areas are too marginal or not present.

Daytime photogenic tornado: a relatively complete tornado structure consisting of a funnel and/or debris from ground to cloud base, clearly visible for at least 180 degrees surrounding the tornado.

Good structure: well-developed and symmetric supercell structure, including high-contrast striations, banding and inflow tails.

Good terrain: flat with few trees or hills (western Kansas)

Mixed terrain: mostly flat with few trees and hills (eastern Kansas)

Poor terrain: mostly trees and/or hills, few views (eastern Oklahoma)

Event-Level QI Formula (QI-E):

1.) Start with Speed multiplier (greater for slower-moving storms) = 60 minus the storm speed in knots

2.) Go through the following for all primary targets total:

Count of all daytime photogenic tornadoes, x 150
Total time in minutes that photogenic tornadoes were visible from a reasonable chase position x (speed multiplier)
Total time in minutes that good storm structure was visible from a reasonable chase position, divided by 60, then x (speed multiplier)
Count of tornadoes visible from typical chase positions x 15
Tornadoes only visible in HP notch x 5
Good lightning = 15 (no multiplier)
Terrain factor (no multiplier):
--Happened in good terrain = 50
--Happened in mixed terrain = 0
--Happened in poor terrain = -50
Road factor (no multiplier):
--1-mile grid, mostly gravel or paved = 50
--2-mile grid or less, few paved = 0
--No grid, few to no paved secondaries = -50
Forecastability (no multiplier):
--Synoptically evident days in advance = 50
--Evident in morning-of data = 0
--Not evident until less than 3 hours prior to start = -50
Rare feature factors:
--Number of instances of multiple simultaneous tornadoes x 40
--Number of instances of tornadoes directly sunlit x 30
--Number of instances of audible roar x 30
Negative factors:
Problematic chaser traffic = -100
Number of tornadoes that caused significant damage x -70
Number of tornadoes that caused fatalities x -200

3.) Redo the above for secondary targets, then divide the secondary target sum by 3.

4.) Add primary and secondary target total, then divide this by the number of primary targets.

5.) Divide this by 100 for the final QI.

Storm Level QI formula (QI-S):

Sum of the primary target portion of the formula above, divided by 100.

I'm leaving out storm mode and supercell type, since it can be assumed most by-definition photogenic tornadoes are going to be with classic supercells. Lower-visibility tornadoes in HPs are already accommodated in the above formula.

 

[JeremyS]

Hey Dan- Any consideration to the tornado outbreak in southern Minnesota 6/17/10? First day I ever saw a tornado and I believe I saw 6 that day. All visible tornadoes of all shapes and sizes. I saw tornadoes for almost 3 hours straight pretty much!

 

[Drew Terril]

5/25 was nice, but I'll disagree massively on it being the better day vs Dodge City, the hand-offs, the 3+ photogenic tornadoes, the multiple tornadoes down at the same time.

I think that's where subjectivity comes into play. Seeing a single tornado on the ground for an hour and a half IMO is a once in a chase career thing, but we see storms drop multiple tornadoes every 5-10 years or so. Maybe not to the extent of DDC, but having missed both, I would have much rather seen Chapman than DDC. I personally thought that was much more impressive from a duration and being able to maintain that type of strength for that long. I don't know if I'll ever have another shot at a Chapman. I think I'll have a much more likely chance seeing a DDC type storm again. Just my opinion, but again, that's where subjectivity comes into play.

 

[Dan Robinson]

This weekend I'll make a web page with a javascript calculator that everyone can play with - both by manipulating the different multipliers and factors and entering data for chase events.

Hey Dan- Any consideration to the tornado outbreak in southern Minnesota 6/17/10? First day I ever saw a tornado and I believe I saw 6 that day. All visible tornadoes of all shapes and sizes. I saw tornadoes for almost 3 hours straight pretty much!

I didn't chase this event, but it should definitely rank high based on what I know about it. To rank the event and each storm, I would need:

Total daytime photogenic tornadoes.
Total tornadoes that were visible to chasers.
Total time that photogenic tornadoes were visible to chasers.
Was the lightning good (frequent/close/photogenic)?
Was structure good? How long did it last?
Storm speed in knots.
Terrain (good, mixed or poor)
Road network quality
How evident was the forecast?
Did the tornadoes cause significant damage, injuries or deaths?
Were there any rare features (two or more tornadoes at once, horizontal vortices, sunlit tornadoes, audible roars)?

 

[Dan Robinson]

I did some more work on the QI-S formula this afternoon using some of the expanded metrics. Here is the spreadsheet formula, Ill call this version 2:

=ROUNDUP(((C2*150)+(J2*50)+(K2*50)+(L2*50)+(M2*30)+(N2*30)+(O2*30)+(P2*-75)+(Q2*-75)+(R2*-150)+(S2*-500)+(D2*15)+(H2*30)+(I2*5)+(E2)*(60-G2)+(F2/30)*(60-G2))/100)

And the results of this second-run analysis:



I created a javascript calculator on my web site where you can play around with the different factors and multipliers used in the formula that control how much weight each metric has in determining the final QI:

http://stormhighway.com/qi.php

You can also download the OpenOffice spreadsheet file here:

http://stormhighway.com/qi/quality-storms.ods

I'm finding it tricky to deal with the human impact metrics. I think the QI should take a big hit for events that cause deaths, injuries and damage, but working that out here hasn't been very easy as you can see.

My thoughts at this stage are to raise the various multipliers so that the top-end events like Dodge City approach 100, which would give some of the lesser events some room to avoid being knocked below zero by negative factors.

If any of you would like to do some experimenting with the numbers and let me know your thoughts, it would be helpful!

 

[Dan Robinson]

After some further adjusting today, I am making these changes:

- Increased the weight of photogenic tornadoes, tornadoes, lightning and rare features.
- Added "historic" metric for a record-breaking event or remarkably rare tornado(es) configuration
- Added an anticyclonic tornado metric
- Added a "major disaster" metric, defined as a tornado that destroys more than 1/3 of a town and/or causes more than 10 deaths.
- Added a final result called "QI-S base" that shows what the result would have been absent human impact factors.

Here is a screenshot of the spreadsheet output with these changes (I'll call this v3):



I'll update the web page and calculator later today.

 

[Dan Robinson]

Version 4 run:



v4 is now live on the calculator:

http://stormhighway.com/qi.php

 

[Warren Faidley]

Interesting index. Where do you get all that time for such fun? I've always based the quality of severe events strictly by the photographic / video opportunities, but that's likely because of my photography background. Campo would rate a solid ten, although I was not there. Had Campo been shot in the early 1980's (producing a good transparency), it would have been close to a 500k + income tornado over the next 20 years. It was an obvious forecast if you were in the region and the road was perfect. It remained on the ground long enough to capture with good contrast. I guess the index depends on the reason behind being there, to just witness a tornado or have serious designs on filming it.