CIPS Experimental Analog-Based Severe Probability Guidance

[Alex Elmore]

Saint Louis University-CIPS has recently developed experimental analog-based severe probability guidance, which is available here: http://www.eas.slu.edu/CIPS/SVRprob/SVRprob.php

There is a link at the bottom of the page that provides a brief technical explanation of the guidance, and we’ll be adding another link soon that provides a video tutorial of how to use the product. In the meantime, here are a couple pointers on how to use it and things to remember:

1. This is different from the current CIPS analog system. The current system is frequency based (i.e., how many of the top analogs are showing severe). This new guidance's probabilities are produced via a logistic regression model that is trained on the top 100 matching analogs and their related SPC severe reports (or lack thereof).
   
   
2. Yes, we are aware of the numerous issues that come with using storm reports for building the guidance and especially using filtered reports for verifying the guidance. A verification page is being developed and should be available soon.
   
   
3. The Day 1-8 guidance maps can be selected at the bottom of the map. The calendar date listed below each “Day #” is the valid date (always at 0000 UTC). Clicking on the calendar date will bring up a page that shows all the guidance that is valid on that date. Once a valid date has passed, the filtered SPC reports will be displayed along the bottom row.
   
   
4. You can access the last 8 forecast dates by clicking on the dates along the top of the map. This allows users to view how the guidance performed over the last 8 days worth of events. We had a network outage two weekends ago, so some of the maps for events over the last 8 days may not be available.
   
   
5. The readout on the right-hand side is based on a selected point on the map, which is done by clicking on the map. Each item in the readout is a clickable link that will bring up a short explanation of what the item is. The probability seen in the readout corresponds to the “raw” probability, so the values may not always correspond to the smoothed contours in the map. In the top right of the map, users can switch between the smoothed contours and the pixel view, which shows the raw probability by grid point.
   
   
6. The guidance is always based on the GEFS mean forecast initialized at 0000 UTC and is always valid at 0000 UTC on the valid date. We are looking into using individual members rather than the mean.
   
   
7. The original purpose of the product is to provide probabilistic guidance in the extended forecast period (3 or 4-8 days out). The guidance is provided in the short term forecast period in order to see trends and for verification. Given the GEFS is a relatively courser model compared to hi-res models available in the short-term (HRRR, NAM 3km, etc.), and the guidance uses the 0000 UTC initialization forecast that’s valid 24 hours later, the guidance may not be able to pick up on mesoscale features that may be significant drivers for severe weather in the Day 1 range. The point of the guidance is to pick up on the synoptic pattern in the extended range, not mesoscale features in the short range.
   
   
8. THE GUIDANCE IS EXPERIMENTAL. There are still things we need to check and adjust if necessary. The “All Severe” probability is what we originally started producing a year ago. The individual hazards (hail, wind, tornado) are recent additions that also need to be evaluated thoroughly.
   
   
9. THE PROBABILITY OF SEVERE IS WITHIN 110 KM OF A GRID POINT. SPC’s probabilistic outlooks are the probability of severe within 40 km of a grid point. Therefore, the products are not directly comparable. We have talked and worked with SPC on the guidance regarding either producing the CIPS guidance on a 40 km grid or figure out how/if the CIPS guidance probability thresholds relate to SPC probability thresholds in order to make it more useful to SPC forecasters.

Sorry for the long post. Feel free to ask any questions below.

 

[Quincy Vagell]

Thanks for sharing this! I saw a few of these graphics posted on social media recently and was really interested in the data if it was available in real time.

I think this is a great tool to use in conjunction with other medium range guidance to get a "feel" for the potential with a given setup. Using the GEFS over the OP GFS is an important distinction, as by using the mean, this can help eliminate issues from fluke/outlier operational runs.

There are caveats with all guidance, including, as mentioned, using this guidance in the short-range. Similarly to the coarse CFS, that data should also be taken with a grain of salt in the short-range. Another consideration is when using this tool in the longer range (especially beyond day 5), if the GEFS mean differs substantially from the EPS (European ensemble), then extra caution should be used. As noted, this system is most likely not going to pick up much, if any mesoscale details, but it is going to be helpful with synoptic pattern recognition.

In my opinion, analog data is an underrated tool when it comes to severe weather prediction. When analogs show a strong signal in one direction (whether it be high "probability" of severe weather or low probabilities), more often than not, that signal ends up being a precursor to the eventual outcome. There are outlier events, but when not dealing with a highly anomalous setup, an analog approach to severe weather prediction has been very useful. Analogs are certainly useful with anomalous events as well, but once there are less "close matches," then it gets a bit more complicated.

Nonetheless, I'm looking forward to using this new tool and assessing its performance going forward. Being able to go backward in time and see how the ensemble "performed" is also very helpful.

Thanks again!

 

[Alex Elmore]

Another consideration is when using this tool in the longer range (especially beyond day 5), if the GEFS mean differs substantially from the EPS (European ensemble), then extra caution should be used.

To build on that, we have noticed that the signal "locks in" around day 5-6. Member spread beyond that range in any ensemble is obviously going to increase to a point that "washes out" the mean, so that's what we're attributing it to. There are ways to address that issue that we are going to look into, but in the end, the guidance is only going to perform as well as the GEFS. It would be nice to use the EPS as well, but we currently lack the funds to do so.

 

[Rich Thompson]

The CIPS analog probabilities have interesting potential. However, as Alex emphasized, the numbers are for a larger grid than SPC forecasts (within 110 km for CIPS vs. roughly within 40 km for SPC). The bigger verification area for CIPS should result in substantially larger probabilities, though the numbers are likely dampened by the coarser nature of the GEFS forecasts. We're still trying to figure out an appropriate calibration compared to SPC forecasts in the extended range of 15% or more.

 

[Jeff Duda]

Is there a peer-reviewed paper on this method? I checked the AMS journals page and didn't see any specific mention of CIPS analog forecasting or using analogs to forecast severe weather. This method strikes me as highly innovative and worthy, and should definitely be tested and documented in permanent literature. If you need a place to start looking for similar references, I would go to Thomas Hamill's work in the mid-2000s. Specifically, this paper is a good one to start with: https://journals.ametsoc.org/doi/full/10.1175/MWR3237.1

 

[Alex Elmore]

Is there a peer-reviewed paper on this method?

Not yet, but we are working on putting one together. The work by Hamill in that study and others by him and his coauthors regarding analogs has been referenced and used numerous times throughout the development of this particular product and the other CIPS analog products.

 

[Jeff Duda]

The work by Hamill in that study and others by him and his coauthors regarding analogs has been referenced and used numerous times throughout the development of this particular product and the other CIPS analog products.

Sounds like an excellent passage to include in the Introduction/Background section

 

[Shane Young]

Also, is there any way (or coming) to access older forecasts? I see the past 10 days are linked, but don't see any menu to go earlier, and changing the URL date didn't net success. Certainly it's probably not your #1 focus to make all that easily available with all you are working on, but would help in weighing the utility of your products! Is it a matter of limited server space or resources!?!

 

[Alex Elmore]

Also, is there any way (or coming) to access older forecasts?

We've discussed having a user-friendly archive of the last 30 days. Im not sure how soon that will happen, but currently, you should be able to view the last 30-ish days worth of forecasts by adding "?rundt=YYYYMMDD00" at the end of the link; the year, month, and day being the date you want to look at. I just tried it and was able to look up the forecast based on the 2018042900 run.