2018 Tornadoes, High Risk, Contest Rules and Ideas

Mark Blue

Staff member
Supporter
Feb 19, 2007
2,756
307
21
Colorado
I was thinking it would be nice to make this a contest where the winner not only received bragging rights for one year, but also gets a $100.00 gift card to Amazon, Newegg, or an online store of their choice. This would be on my dime, unless @Steve Miller wants to split it with me.

However, we need a good set of rules on how to determine the winner, plus a deadline for entries to be made. I’d also like to see a points system where going over on the tornado count or the first high risk date doesn’t disqualify the participants and a way to handle points if someone guesses no high risk days. I think adding the state might over complicate things, but am open to keeping it if it can be quantified with points. What does everyone think? Please throw out ideas on the rules if you’ve been down this road or have a bright mind for mathematics. I can split this into another thread if it results in a lot of discussion.
 
  • Like
Reactions: Brandon Centeno
Jul 5, 2009
806
469
21
Newtown, Pennsylvania
How about lowest number of points wins. One point for every number of tornados off from the actual total (absolute value, so whether over one or under one, you get one point); one point for every number of days off from the high risk date (five days before or after your guess, you get five points). If you guess no high risk days and there is one, you get a flat number of points added to your total. If you guess the state for extra credit, you get a flat number of points deducted. How many for each of these? Not sure, but figured I would put a few thoughts out here that others can add to [emoji848][emoji57]


Sent from my iPad using Tapatalk
 
  • Like
Reactions: Mark Blue
Jun 16, 2015
407
867
21
32
Oklahoma City, OK
quincyvagell.com
How about lowest number of points wins. One point for every number of tornados off from the actual total (absolute value, so whether over one or under one, you get one point); one point for every number of days off from the high risk date (five days before or after your guess, you get five points). If you guess no high risk days and there is one, you get a flat number of points added to your total. If you guess the state for extra credit, you get a flat number of points deducted. How many for each of these? Not sure, but figured I would put a few thoughts out here that others can add to [emoji848][emoji57]


Sent from my iPad using Tapatalk
If we do something like that, how about those who made a prediction earlier have less total points as well? Maybe something like a deduction equal to the difference between the number of days between the deadline and the entry.

For example, if the contest closes on March 31st and long range data suggests the potential for a higher end event around April 7th, there won't be nearly as much of a advantage compared to someone who has already submitted a guess.
 

Jeff Duda

Resident meteorological expert
Staff member
Oct 7, 2008
2,954
1,430
21
Westminster, CO
www.meteor.iastate.edu
Mark, if you're willing to foot the bill, the rules are yours to decide upon. But my suggestions are:
-compute the absolute tornado count error (so |actual tornado count - estimate|) and scale the high risk issuance so that a miss on that represents the climatological spread of seasonal tornado counts. I don't wanna take the time to compute that right now, but you would follow the methodology here to adjust the seasonal counts to account for the long term trend, then compute the standard deviation of the adjusted yearly counts. I'm guessing the standard deviation is somewhere around 150, so you could scale the high risk error to have about the same value as being off on the tornado counts by 150.

-I wouldn't worry about creating a point value for the "no high risk" category because if there is a high risk, there's no reason someone who predicted "no high risk" should win, even if they get the tornado count right on. And if there is no high risk, then there's no reason someone who predicted a high risk should win, even if they get the tornado count right on.
 
  • Like
Reactions: Mark Blue

Mark Blue

Staff member
Supporter
Feb 19, 2007
2,756
307
21
Colorado
I was going to say that if for some odd reason I won, I’d defer the gift card and give it to the person in second place. As far as tornado counts we’d have to wait until early January 2019 for the final SPC number right?

I like everyone’s ideas here. My biggest concern right now is this: is there enough time to get it up and running if the deadline is March 31st for entries? I also hope I’m not stealing the OP’s thunder by commandeering his thread and turning it into something more than he intended. What do you guys think?
 

Jeff Duda

Resident meteorological expert
Staff member
Oct 7, 2008
2,954
1,430
21
Westminster, CO
www.meteor.iastate.edu
As far as tornado counts we’d have to wait until early January 2019 for the final SPC number right?
Technically the final numbers are to be derived from Storm Data, which is not usually complete until about 3-6 months after the fact. So, realistically, we would have to wait until next spring. 2017 data is only complete through November at this time, if that gives an indication of how long after the fact it takes to get the final numbers.

Or you could use SPC's numbers, but be sure to specify that in the rules and give those who have already guessed a chance to change their guess to account for the fact that SPC numbers are always going to be high due to duplicated reports. SPC uses an approximation of 85% to derive final values beforehand, so that could also be a condition.
 

Mark Blue

Staff member
Supporter
Feb 19, 2007
2,756
307
21
Colorado
I think I’m going to go ahead and award a winner this year based on some form of points as described by James above. It’ll be a test run of sorts to see how it turns out, then next year we can refine the rules based on the other feedback in this thread (or if we receive anymore during the interim) and tighten things up a bit to make it as fair as possible.

Knowing then what we don’t know now will enable us to start the contest earlier with a set of posted rules and run a more organized contest that’ll hopefully draw several more entries than we have this year. Thank you everyone for being a sounding board and giving me so much food for thought.
 
  • Like
Reactions: Todd Lemery

Mark Blue

Staff member
Supporter
Feb 19, 2007
2,756
307
21
Colorado
We’ll my High Risk prediction of 4/27/18 is looking like I totally blew it. It isn’t even May yet, so until we see some moisture making its way poleward I’m thinking we’ll be lucky to have a High Risk event this year.

Still need to finalize the contest rules as well....
 

Jeff Duda

Resident meteorological expert
Staff member
Oct 7, 2008
2,954
1,430
21
Westminster, CO
www.meteor.iastate.edu
There are going to be a lot of high risk busts...

I was going to put together a graphic showing the distribution of forecasts. Guess I better get on that...stay tuned.

(EDIT/ADDED):
No graphic, as sample size was rather low, but here's a text description of the forecasts.

N=37
Mean / median tornado count: 1062 / 1047
(conditional) Mean / median day of first high risk: May 6 / April 29
The condition above results from 5 predictions of "no high risk"

What is very interesting about the predictions this year is that the location of the distribution is greatly different than climatology. The average number of tornadoes in the US over the last 50ish years is ~1300. Only two predictions were for an above average count. A substantial fraction (14/37 = 38%) was for less than 1,000 tornadoes.

Also, 11 forecasts of the first high risk have already been too early, with another 6 virtually certain to also be too early. Only three forecasts called for a first high risk to occur after the end of May, and half of the forecasts for the date of the first high risk were before May 1st.

Let's see how this shakes out.
 
Last edited:
  • Like
Reactions: Todd Lemery

Jeff Duda

Resident meteorological expert
Staff member
Oct 7, 2008
2,954
1,430
21
Westminster, CO
www.meteor.iastate.edu
Update: As of May 4th, 20 of the 37 predictions include a high risk that has been too early, and an additional 5 (which predicted the first high risk within the next week) will almost certainly verify also as too early.

It's looking good for those who went later than mid-May for the first high risk...
 

Jeff Duda

Resident meteorological expert
Staff member
Oct 7, 2008
2,954
1,430
21
Westminster, CO
www.meteor.iastate.edu
Update: Only one person who predicted there would actually be a High Risk hasn't yet "busted" on the date of the first high risk. So if you were the one person who selected the June date for the first High Risk or the November date, you two are basically the only two that can win that category unless there are no High Risks. And honestly, at this point, the chances of "no high risks" seems to be substantial, although all it would take would be one big fall event to bust that, too. If I were to do a fivethirtyeight-style analysis of the probabilities, I would say the person predicting the November High Risk has the greatest probability at this moment of winning the category.

As far as counts go, most predictions were for a below-normal season, so most predictions are still well within uncertainty bounds. But if you went for the low end of the distribution (especially below 1000 tornadoes), you should be feeling pretty good at this point.
 

Mark Blue

Staff member
Supporter
Feb 19, 2007
2,756
307
21
Colorado
Excellent analysis Jeff. I think the High Risk ship has sailed for 2018 but my predictions were so far off it doesn’t matter what I think!
 
Jun 18, 2017
40
19
11
Tennessee
Well, might wait and see what happens in November (and Dec?). Lower chance though but there were a few high risks issued in both months since 2000.
 

Jeff Duda

Resident meteorological expert
Staff member
Oct 7, 2008
2,954
1,430
21
Westminster, CO
www.meteor.iastate.edu
Update: at this point it seems rather improbable that a high risk is coming, so that leaves 5 people who are going to score pretty well in that category. The inflation adjusted tornado count stands at 897. I'm sure that will increase some throughout the rest of the year, but it's looking like there are three people who will be competing for the win. These three people went with "no high risk" and a tornado count of 875, 950, and 967.