SN rolls out a Ranking system

SIGH... Seems I'll be stuck at where I am for awhile, I'm active when storms come through but it seems that so far our storms have been below severe strength here (luckily). I see that as one flaw for the ranking system at least from my perspective.
 
Tyler:

Seems like a decent idea. My first "gut" reaction would be this may promote unnecessary reports, like many folks reporting the same event, just to earn the points. Perhaps the system should be based on a ratio rather than actual reports. Also, it will be interesting who this will affect work load for you and others.

But again...this is just a first...gut reaction. I personally am not to concerned what my rating is really...the point is for the right people to get the information, but I see how this might help with habitual offenders.
 
I haven't ever heard the National Weather Service complain they have gotten too many reports from a storm. I think that's a bridge that should be crossed if it becomes a problem :)
 
I am curious, who determined if the reports are good, bad, or ugly? I see that Verne Carlson and David Drummond both have 'Very Bad Reports' in their list. However, even though I don't personally know either one of them I have seen enough of their work to be pretty sure they didn't report a "sheriffnado" or "extreme lightning".

Maybe it was a case of not providing enough information. That being said, when you are out chasing (especially if you are alone) sometimes all you have time to do is just click the boxes on what you see. There's no time to write and essay, no place to pull over safely to do so, it's hard to tell which way the tornado is moving or how fast, etc... But I've seen some reports given bad grades for not having enough detailed information. I thought that was the purpose of having the phone number on our profiles, so the NWS can call us for more information if they want it. It's a lot easier to talk into a Bluetooth headset while driving than to type out a detailed report. And some people might be less inclined to make reports if they are going to get dinged a point for not typing out that essay while trying to navigate safely through a tornadic supercell.

Edit: I just looked at my SN account. I got a red flag and a yellow flag for the May 10 tornado near Medford, OK. I wrote this while driving East on SR 11 with a possible rain wrapped tornado to my left. (So I really should not have typed anything.)
"approx 5 min ago tornado crossed sr 81 heading ene" I entered my position as 1 mile North of Medford, OK. I know that it would be great to have more detailed information, but see see the above. There were several reasons I didn't give more precise distance. 1. I was driving and the storm was moving at 60MPH. 2. It really is hard to judge distance. 3. The warnings don't get down to specific lots and houses, and the hook can be seen on radar. Is it going to have any benefit whatsover to say that it was 1 mile North of my position? If I overestimate or underestimate that and if hypothetically the NWS did alter their warning it could be more damaging than just issuing a warning based on reports and using the radar data to provide an estimated path. I thought the purpose of this grading system was to combat the people who are just throwing silly reports out, not to penalize people who are actually trying to do something good but sometimes don't add info because they don't have it or cannot safely provide it.
 
Last edited by a moderator:
Wes Carter said:
I am curious, who determined if the reports are good, bad, or ugly?
Click to expand...

The QC team (names listed on our "About" page)

I see that Verne Carlson and David Drummond both have 'Very Bad Reports' in their list. However, even though I don't personally know either one of them I have seen enough of their work to be pretty sure they didn't report a "sheriffnado" or "extreme lightning".
Click to expand...

Actually you bring up a good point. I used incorrect terminology and have since corrected it. "Very Bad" now reads "Low Quality".

Obviously, our quality control team is not there (usually) so reports are based on the "quality" of the report.

There are three categories:

- High Quality / Perfect Report [ green flag ]
- Acceptable Quality [ yellow flag ]
- Low Quality [ red flag ]

Maybe it was a case of not providing enough information. That being said, when you are out chasing (especially if you are alone) sometimes all you have time to do is just click the boxes on what you see. There's no time to write and essay, no place to pull over safely to do so, it's hard to tell which way the tornado is moving or how fast, etc... But I've seen some reports given bad grades for not having enough detailed information. I thought that was the purpose of having the phone number on our profiles, so the NWS can call us for more information if they want it. It's a lot easier to talk into a Bluetooth headset while driving than to type out a detailed report. And some people might be less inclined to make reports if they are going to get dinged a point for not typing out that essay while trying to navigate safely through a tornadic supercell.
Click to expand...

Nobody is asking for an essay, in fact you might get a "low quality" report if you write an essay for no good reason.

If all you click is "estimated winds 50mph" you are going to get an Acceptable Quality report. It doesn't take but another 5 seconds to type: "1 inch tree branches snapped off". That makes it a High Quality report.

And let's turn this around. You can't expect the NWS to call every person who submits a report, especially to just clarify a small detail.
 
Wes Carter said:
Edit: I just looked at my SN account. I got a red flag and a yellow flag for the May 10 tornado near Medford, OK. I wrote this while driving East on SR 11 with a possible rain wrapped tornado to my left. (So I really should not have typed anything.)
"approx 5 min ago tornado crossed sr 81 heading ene" I entered my position as 1 mile North of Medford, OK. I know that it would be great to have more detailed information, but see see the above. There were several reasons I didn't give more precise distance. 1. I was driving and the storm was moving at 60MPH. 2. It really is hard to judge distance. 3. The warnings don't get down to specific lots and houses, and the hook can be seen on radar. Is it going to have any benefit whatsover to say that it was 1 mile North of my position? If I overestimate or underestimate that and if hypothetically the NWS did alter their warning it could be more damaging than just issuing a warning based on reports and using the radar data to provide an estimated path. I thought the purpose of this grading system was to combat the people who are just throwing silly reports out, not to penalize people who are actually trying to do something good but sometimes don't add info because they don't have it or cannot safely provide it.
Click to expand...

The SN advisors accept that we have not been consistent in the past. We have formed an internal group to come up with a consistent rating system.

It also takes 4 or more "red" flags to get ranked as a low quality report. That allows for variability of opinion between humans :)

-Tyler
 
Not to beat a dead horse, but since I feel SN is a great tool (if the local NWO used it) but like I posted on the support forum I think training tool on exactly how to report since it seems that the same things keep getting tossed. I know that there is the training and it is a great tool to help weed out exactly what severe is, but maybe a module that focuses on better reporting?
As a back on focus, last year my county was one of two in the ARX area that did not have any activations for spotting, that severely lowers my reporting cred, however locally I am one of the few that does go out and reports and usually submit over the local repeater, but being the Skywarn Director I'm trying to encourage a way for folks to track where everyone is and like SN for that feature so I hope you can figure out a way to help folks like me out.
Thanks for a great system, and I think this will help eventually once the few problems are ironed out.
 
Wes Carter said:
Edit: I just looked at my SN account. I got a red flag and a yellow flag for the May 10 tornado near Medford, OK. I wrote this while driving East on SR 11 with a possible rain wrapped tornado to my left....(deletia)
Click to expand...
Wes, we really appreciate the reports you provide via SN, but I do have to make a comment here which applies to all users, so I'm not just singling you out...

If a SN observer is *driving*, and can't pull over to type in the report, nor has a passenger who can type in the report, then don't use SN to make the report. You are putting yourself (and other drivers) in danger with this kind of multi-tasking. If you absolutely feel obligated to be able to make spotter reports all the time, even when chasing solo and in the act of driving, employ a redundant alternative means, like a bluetooth phone (as you suggested) or the old standard, a ham radio. Then, the folks you relay the report to can ask you any additional qualifying questions to be sure your report is complete enough to base warning decisions on, and you can a hand on the wheel and your eyes on the road.
 
Tyler Allison said:
There are three categories:

- High Quality / Perfect Report [ green flag ]
- Acceptable Quality [ yellow flag ]
- Low Quality [ red flag ]
Click to expand...

Question about this. According to my "ranking" I have submitted 8 "acceptable" reports and 1 "perfect" report yet when I go back and look at my reports, 8 out of the 9 are all green flags (one yellow flag on one report when I accidentally clicked non-rotating instead of rotating wall cloud). Considering all the other reports have all green flags, shouldn't they be "perfect" reports, while the one with the yellow should be "acceptable"? If that is the case, I should have 8 perfect and 1 acceptable, instead of 1 perfect and 8 acceptable as it is listed now.
 
I'm another one of those guys who has always done his reports via ham, and only this year started using SN for position reporting. I posted in another thread asking whether I should start repeating my reports by text on SN.

Question for Tyler.....What do you think about adding a box we can check that says something like "This was submited by ham radio", or "copy of ham radio report", or ????

BTW, I'm a huge fan of SN. A great public service for both the spotters, weather service, and the county EOC managers.
 
I got 100% on the test on the first and only time I took it. Made one report about sub severe winds to let the NWS know the part of the storm I was on had not produced a severe wind gust in over an hour, and now my reporting priveleges are suspended? I guess to some no report is finally better than some report. If that's the way it's going to be, then good riddence SN.

I should also add that I haven't not done a ton of reporting on SN so this one "red flag" report really brought down my ranking.
 
Last edited by a moderator:
Jason Foster said:
Tyler:

Seems like a decent idea. My first "gut" reaction would be this may promote unnecessary reports, like many folks reporting the same event, just to earn the points. Perhaps the system should be based on a ratio rather than actual reports. Also, it will be interesting who this will affect work load for you and others.
Click to expand...

That might be true -- but is that a bad thing? 20 duplicate tornado reports simply reinforces that there is, indeed, a tornado happening.

On the other hand, if others report a tornado that they can't see simply because they see someone else reporting it and would like some "points", that's bad.
 
Dean Baron said:
I got 100% on the test on the first and only time I took it. Made one report about sub severe winds to let the NWS know the part of the storm I was on had not produced a severe wind gust in over an hour, and now my reporting priveleges are suspended? I guess to some no report is finally better than some report. If that's the way it's going to be, then good riddence SN.
Click to expand...

Hi Dean, don't jump to a big conclusion there. Because of the type of system SN is, there has to be minimum standards for reporting and reporting of sub-severe information just would fill up the system with all kinds of useless info with reports like 'not much wind' or 'it hailed'. I agree that sometimes the absence of severe weather in a location is reportable, but not in SN. We've all seen the reports of 'wind', minimum standards have to be set. Yes, yes, yes, sometimes a 'negative' report saying something isn't there can be extremely valuable, but how do you do that in a large system? Saying 'use common sense' just won't cut it unfortunately as we all know that ran out years ago.

So, as you start with two points, then took the training to get two more, and with the bad report, it drops you back to 2 points. You will need to send an e-mail to Tyler to be re-enabled, and I hope that you do.
 
Last edited by a moderator:
I believe SN is the same as ham radio nets that are SPECIFICALLY under Skywarn where NWS has directed not to report non-severe conditions.

Hell I saw tight rotation today...but by the time I got SN up and ready to make a report the rotation dissipated. The storm was already tornado warned by doppler, so my add would have done nothing to increase the current warning. Will I submit some report directly to NWS after the fact...maybe, for no other reason to verify rotation, but I'm sure the local WFO webpage has a direct report submission page for that kind of thing, which I believe is better for these none severe reports.

Bottomline: don't worry about the ranking system or correcting others on SN...prioritize your reporting as to the importance of life safety and property protection.
 
Was just going to make a comment like the last couple.. Yesterday as we were heading out for supper the wife's phone alerted us to a severe storm warning for my county, and since we were heading towards the storm I decided to station myself at a spotting location and see what happened. Well, the storm decided to drop 1/4 inch hail on us for a brief couple seconds and since ARX in their hazardous outlook requested any hail reports for the day I reported it to the NCS. I did not send it through SN even though I considered it as I didn't want to get penalized for sending a report that didn't meet severe criteria even though the local office wanted that. How would I best handle a case like that? As it was that was one of two hail sent in, but they did get a civilian report with hail laying 2" deep, but didn't get my report as they didn't have anyone on their end monitoring the local repeater. (Will be addressed at our next Skywarn meeting).
Thanks ahead of time for clarifying my question.
 
PKohlmeier said:
I did not send it through SN even though I considered it as I didn't want to get penalized for sending a report that didn't meet severe criteria even though the local office wanted that. How would I best handle a case like that?
Click to expand...

You did the right thing.

If your local NWS office is asking you to send in sub severe reports you should do so. But not via the Spotter Network. If I was so inclined to submit non-severe weather conditions I would utilize the SN to obtain the phone number for the office and call them. Or participate in the CoCoRaHs network which is designed for such observations.

It is unfortunate that NWS offices do not abide by the national standard and thus place the rest of us in a predicament of supporting 140+ various reporting criteria or the official one. We have decided to support the official one.

-Tyler
 
Dean Baron said:
[...] Made one report about sub severe winds to let the NWS know the part of the storm I was on had not produced a severe wind gust in over an hour [...]
Click to expand...

I agree with this. Maybe there should be an option that allows users to input a report as "sub-severe" criteria. A storm being warned for "destructive winds in excess of 80MPH," but is consistently producing only 50-57MPH winds would be rather notable (this is sometimes the case when a storm rooted in elevated instability). Same applies for hail, tornadoes, etc..

It would be nice to get near real-time ground truth in a storm's environment without going through the hassle listed above (calling, CoCoRaHs, etc.).
 
Last edited by a moderator:
You must log in or register to reply here.

Similar threads

Bob Fanucchi
SN SN Sign in issues. 2024 APR 23 @ 11:27:05
Replies
2
Views
196
John Wetter
John Wetter
Jake Thompson
Unable to load SN Reports Feed
Replies
0
Views
708
Jake Thompson
Jake Thompson
John Wetter
  • Sticky
SN 2022 Spotter Network Newsletter
Replies
23
Views
4K
Lou Ruh
Lou Ruh
Jesse Risley
  • Sticky
The State of Stormtrack: Fall 2023 Update
Replies
1
Views
1K
Shawn Camp
John Wetter
  • Sticky
SN 2021 Spotter Network Newsletter
Replies
0
Views
2K
John Wetter
John Wetter
Back
Top