The Foibles of Long-Range Models (and Why Many Chasers Use Them Anyway)

[Bob Hartig]

At 300 hours out, Tuesday looked like it had potential. Today it looks non-existent.

Is that any surprise? No, of course not. Anyone who's been forecasting for a while knows that beyond three days out, numerical models move increasingly from the realm of forecasting to that of fortune-telling. The long-range GFS in particular is notoriously unreliable. But many of us peek at it nonetheless. We can't help ourselves, especially at the end of a long winter, and especially those of us who live a long distance from Tornado Alley.

This post isn't intended to discuss the merits of the ECMWF over the GFS, or why the one is gold and the other garbage. Rather, it's meant to make newer members aware of one of the considerations involved in chase forecasting, and to give those of us who've been here a while a chance to sound out on a topic that inevitably resurfaces now and then.

Here is my blog post on the matter, with a couple of forecast maps to illustrate the point.

 

[rdale]

What did the ensembles say

 

[Marcus Diaz]

Well I admit I do look at the long range GFS up to a point. Mainly I'm looking for troughs to dig into the south, and watch the jet stream patterns. If anything its a quick way of saying "gotta watch this system". I've noticed really powerful troughs from around the 300 hour mark be somewhat reliable in terms of it actually happening, but that's merely a very general idea of when and where.

 

[Bob Hartig]

What did the ensembles say

Last I checked, they were saying, "Abandon all hope, ye who enter here."

 

[Jeff Duda]

The real message here is that you shouldn't expect to get a gold-medal winning forecast every time you look beyond a few days (three is arbitrary...it depends on what you're hoping to find). We can thank Ed Lorenz for this notion. As he first wrote about in his 1969 paper that basically introduced chaos theory (see http://onlinelibrary.wiley.com/doi/10.1111/j.2153-3490.1969.tb00444.x/pdf), we can only expect a certain degree of predictability for a given spatial scale. The largest scale features are those for which predictability goes out the farthest. On the other hand, model error grows extremely fast for convection-allowing models. Predictability can be extended to an extent by the use of model ensembles. At least we can get an idea of the uncertainty in a forecast from ensembles. Unfortunately, almost every operational global model ensemble suffers from under dispersiveness, meaning that the actual weather too frequently occurs outside the bounds of the forecasts from the extreme members.

Yes, the GFS may show a statistically poor performance compared to the ECMWF, but that doesn't mean the GFS is not worth a damn. You'd be surprised how often it can still give a useful synoptic scale forecast even a week out. If you're trying to nail down a target city within a three county radius and an afternoon dryline position 7 days out, expect to be disappointed more often than not.

 

[Robert Edmonds]

Just to hammer home Jeff's point... One of the points from my guest post on your blog, if poorly conveyed, is that the atmosphere is chaotic. Small changes in initial conditions will eventually lead to drastic differences in model the model solution. It's not just bad data, but data will always be imperfect and incomplete, so solutions should be expected to diverge drastically after some time. Also, models induce diffusion and dispersion (depending on scheme and problem). That is, models don't solve the equations for fluid flow, but instead something that approximates the equations (which hopefully become better, but not always, with increasing spatial and temporal resolution). I also want to bring up that the reason why the ecmwf may handle long range forecasts better is that it is a spectral model. They naturally handle spatial derivatives better, and have fewer phase issues (which are probably important in long range forecasts). Which brings me to my last point, use the model best designed for the problem you are trying to solve.

 

[MClarkson]

I thought the GFS was spectral as well?

 

[Robert Edmonds]

I thought the GFS was spectral as well?

You're right, where did that mix up come from??? Maybe I was thinking of what I use for Mars that can be run at a similar horizontal resolution as the GFS but is not spectral...

 

[Dan Robinson]

If you have more than 2 runs of consistency *and* agreement with one or more other models, long-range chase planning of synoptic patterns can be performed with at least moderate confidence. If I had to give rough percentages in long-range forecasting for 7-10 days out, I'd probably go something like this:

- More than 3 successive runs of consistency plus agreement between ECMWF, GFS and GFS ensembles = 40% chance of something close to the indicated synoptic pattern actually happening.

- More than 3 successive consistent runs of the ECMWF, regardless of other models = 30% chance of something close to the indicated synoptic pattern actually happening.

Past 4 days, any pattern shown by the GFS, regardless of run-to-run consistency, seems to have generally a 20% chance or less of anything close to the indicated pattern actually happening. For the Euro alone, I'd put this slightly higher, maybe in the 30% range past 4 days.

When there is no run-to-run consistency or agreement between models, you have a better chance of rolling dice to determine the synoptic pattern past 4 days. Past 6-7 days, the GFS is a total fantasy forecast. I think I've seen the GFS get it right past 6 days maybe once or twice a season, but even a stopped clock is right twice a day. Past the ECMWF's range, there is no reliable method.

That's all just my personal average from planning chase trips from West Virginia for many years. Of course, every setup is different. I've seen solid forecasts (lots of agreement and consistency) completely tank 3 or 4 days out, plenty of times. But generally, a big trough shown for more than 3 runs by the ECMWF inside 8 days is a good bet, especially if the GFS/ensembles are on board.

 

[David Wolfson]

... And with Bob's example the ground truth is a vigorous storm over Wisconsin with shorter-wave mid-upper energy embedded in a broadly zonal WNW flow. True, it's a day earlier and did not cut off, like the 300 hr. model forecast, but I'm looking at a model forecast that's remarkably accurate with respect to major features. With respect to severe, just eyeballing the 300 hr. example, IMHO severe is a long shot anyway with an early-season system like this. You'd need a broad mid-level southwest flow early enough to allow energy to advect north before the mid-upper flow blasts the cold front in from the west.

So long as you don't rely on the long-range models to accurately forecast smaller-scale features that far out, I'm constantly amazed at how accurate they are becoming for the large-scale view even two weeks out, provided you average out successive runs over a few days. FWIW.

 

[David Wolfson]

... and along this line, the GFS has been showing a consistent zonal flow which breaks down right about the last day of March (~258 hr. ATTM) with what would be the first traditionally chaseable early-season system for Texas and Oklahoma. Let's see what happens....

 

[Bob Hartig]

True, it's a day earlier and did not cut off, like the 300 hr. model forecast, but I'm looking at a model forecast that's remarkably accurate with respect to major features. With respect to severe, just eyeballing the 300 hr. example, IMHO severe is a long shot anyway with an early-season system like this...So long as you don't rely on the long-range models to accurately forecast smaller-scale features that far out, I'm constantly amazed at how accurate they are becoming for the large-scale view even two weeks out, provided you average out successive runs over a few days. FWIW.

The major features seem to be the separating point, and from my perspective, they're why it pays to keep an eye on the long-range models. If you go by low-level moisture and surface winds, the two examples I provided are obviously worlds apart. But if you go by 500 mb heights and dismiss the matter of timing, then the GFS did a pretty good job. From what I can recall (without actually retracing the model runs out to 300 hours), the progression from farther out to closer in involved some of the usual flip-flopping; and 24 hours IS a significant difference. But that's not really an issue. It's normal. If one has realistic expectations of long-range models, then on that larger scale, the GFS serves a useful purpose in providing a heads-up that something could be on the way...and it appears to be developing some continuity...and now it's coming into the range of mesoscale forecasting...and voila! Here it is, in whatever form it finally takes.

It's a matter of what a person looks for from long-range forecast maps. I think most folks pay attention to the 500 mb chart, which is a broad indicator of the potential for things to come. I check out some of the other stuff anyway, because why not. But I expect things like timing, moisture, wind speeds, and such to change as the forecast hour closes in. Overall, I think Dan Robinson's assessment captures it well.

BTW, to clarify a comment of mine in my original post: I wasn't stating my personal opinion of the ECMWF versus the GFS. Rather, I was recognizing the strong perspectives that some hold on the matter, and I wanted to avoid making such comparisons the thrust of the discussion.

 

[Joshua Nall]

I just need a hint about scheduling some extra work in the short term, so I can take off a day or two the following week. If the big trough showing up on the 500 mb chart doesn't pan out, I'll take that time and go fishing or something That is about the extent of my long range model looking.

 

[Andy Wehrle]

The long-range models are really only good for general patterns, as has been alluded to previously. For instance, right now the GFS shows a possible setup around the April 5-6 timeframe. I don't care about the exact track of the surface low, frontal positions, exact dewpoint temperatures or wind speeds, because those details can and will change dramatically by the time the day actually arrives. All I care about for now is that it shows some "light at the end of the tunnel" with a weakening of this persistent negative AO and a transition to a more spring-like pattern.

 

[Adam Lucio]

I've been paying attention more to the EURO lately than I ever have, and I find it to be just as inaccurate as the GFS. I still don't get why everyone loves it so much. It may wobble around less and show less run to run inconsistency (probably because there is no 6 or 18z whacker run - at least that I know of) but IMO the difference in ones reliability over another isn't that great. Take this latest winter storm for example...the EURO was always the driest of any forecast model...and parts of central IL got 18" of show. Sure the EURO was consistent in its drier solution, but that solution was wrong.

I've also seen both the GFS and EURO do a pretty good job at 300+ hours out. There are quite a few times the models can get a good grasp on the general pattern that far out, but individual systems always vary.