Why are some model runs better/worse than others?

[Andrew Clope]

I've seen this mentioned at different times and wonder why?

Currently I am looking at Sunday in Eastern NE, NE KS and possibly Central MN. GFS looks better than NAM (Likely an entirely different topic of conversation), but I notice that I can pick various model runs eg 12Z 18Z 0Z 6Z 12Z.

Now, I would assume that the closest run to the time of initiation that you're looking for would be the best. But from what a few have said, some model runs are not as well thought of. I guess I'm just wondering why?

Thanks!

 

[Marcus Diaz]

Without sounding at all technical...some runs get contaminated with some bad data that can show a totally different scenario the previous run shows. The thing to look for in consistency. If a model goes all over the place multiple times, you tend to not lean on that one. I'm sure others will put more detail. Don't mind me though, I fix cars for a living :3

 

[MClarkson]

What you may be referring to is the fact that the balloon-carried instrument packs (radiosondes) used for measuring upper air values are typically only launched just before 12z and 00z. Thus the 06z and 18z model runs tend to have older data

 

[Andrew Clope]

Without sounding at all technical...some runs get contaminated with some bad data that can show a totally different scenario the previous run shows. The thing to look for in consistency. If a model goes all over the place multiple times, you tend to not lean on that one. I'm sure others will put more detail. Don't mind me though, I fix cars for a living :3

What you may be referring to is the fact that the balloon-carried instrument packs (radiosondes) used for measuring upper air values are typically only launched just before 12z and 00z. Thus the 06z and 18z model runs tend to have older data

Ah, thanks guys! That makes more sense. So "generally speaking" the 00z and 12z runs may tend to be a bit more reliable due to fresh data.

 

[Mark Ellinwood]

Nowadays the difference between the "on-hour runs" (0z and 12z) and the "off-hour runs" (6z and 18z) is negligible. There were notable differences back in the early 2000s, but now that the models ingest so much data (mostly from satellites), there is little difference in skill. The quantity of model-ingested satellite data >> radiosondes.

 

[Paul Knightley]

Firstly you have to compare eggs with eggs - thus, mesoscale models which can simulate convection (e.g. WRF (HRRR) will quickly give different (local-scale) output to global/larger scale models which only parametrize convection over an area (i.e. they don't simulate the storms themselves, they simply 'switch on' convection when certain conditions are met). So, if you compare GFS with a hi-res model, you're essentially comparing two different beasts - in many case, you might still get the same general features, but often the output will look quite different on a local scale.

Secondly, the way models get their initial conditions is not the same (because there is no perfect method, hence different methodologies!). This 'initialisation' is (probably!) the biggest cause of model differences and errors in forecasts...this can be demonstrated by the fact that the same data (obs, satellite data, etc) can be used, and yet models come out with different solutions - of course, the models' physics will be somewhat different, but I think initialisation is a bigger cause.

 

[John Wetter]

The various spring workshops over the years have shown that the 00Z models had better skill than the 12Z models... If you want to read more into the 'why' of this, I'd suggest googling around for the spring experiment model comparisons; lots of good articles out there. This information is why many chasers started to subscribe to the idea of looking at the 00Z models, but then the day of the chase only look at observational data and VIZ satellite imagery. Many still subscribe to this. The CAM's have changed all that though. You have models like the HRRR, and soon the HRRRE, the HopWRF, and others that are doing explicit convective forecasts on an hourly or three-hour update schedule with ensemble products and it can really help a lot with how a system is evolving.

 

[Mark Ellinwood]

The various spring workshops over the years have shown that the 00Z models had better skill than the 12Z models... If you want to read more into the 'why' of this, I'd suggest googling around for the spring experiment model comparisons; lots of good articles out there. This information is why many chasers started to subscribe to the idea of looking at the 00Z models, but then the day of the chase only look at observational data and VIZ satellite imagery. Many still subscribe to this. The CAM's have changed all that though. You have models like the HRRR, and soon the HRRRE, the HopWRF, and others that are doing explicit convective forecasts on an hourly or three-hour update schedule with ensemble products and it can really help a lot with how a system is evolving.

Usually, the most recent model run will be the most accurate, since skill increases as lead time decreases. I would much rather use 6/12z data vs. 0z the day before or the day of. I'd like to know what data was used to support older model runs having better verification.

 

[Rob H]

Usually, the most recent model run will be the most accurate, since skill increases as lead time decreases. I would much rather use 6/12z data vs. 0z the day before or the day of. I'd like to know what data was used to support older model runs having better verification.

It wouldn't surprise me if a lot of these workshops were working off of research from several years prior, when the quantity and quality of data from non-rawinsonde sources was worse.

 

[rdale]

With the ways models are upgraded, I'd be shocked if someone presented a paper with old data. It'd be useless.

 

[Brian G]

First time poster here...saw this thread and thought I'd chime in.

Of some relevance is the NCEP presentation Justification for the 06Z and 18Z Runs. The summarized points are:

1. The 6Z and 18Z cycles are slightly worse than 0Z and 12Z for the same lead time, but the differences are probably insignificant especially when compared to the much bigger differences relative to other models.

2. For the same forecast time (6Z @ 114 compared to 0Z @ 120) the latest cycle (regardless if it is an "on-hour" or "off-hour" run) is usually better. For a D3 forecast the 6Z cycle is mostly better than 0Z. And for a D1 forecast the 6Z run is always better than 0Z. Again, this is for the same forecast time (not lead time).

 

[Marcus Diaz]

I kinda figured there wasn't enough difference between all the runs to delineate which run is best. It seems like I've seen just about each run get it terribly wrong and each run get it amazingly right. The 6z and 12z runs on the morning of are probably the best info you can look at. I think the 0z run is popular among chasers since that's the last run before most people go to bed or make the all-nighter to the target. And its usually the run people make their decisions on for a rough target the next day. The only thing I notice is more often I see the 6z and 18z become the furthest away from consistency more times than the 0z or 12z. I'm sure there's explanations as to why, but that always seems to be the stigma on the "off" runs.