Which models suck and why?

[JamesCaruso]

Is this a record number of consecutive posts from the same person on the same topic?

This thread reminds me of a question I have always had, and believe me it is not my intent to start any kind of argument or debate about climate change, which is obviously a hot button topic like religion or politics - this is truly offered as a serious question: Is there something different about the longer-range climate and global temperature models that gives us a greater level of confidence than we seem to have with any other models? We all distrust model output beyond 5 to 7 days, so wouldn't the extrapolation and compounding of biases and errors going out years and decades render those long-range climate models completely worthless? Or are those models simply a different animal, i.e., maybe they are large scale and global, and focused on fewer parameters, actually leaving less room for error as compared to a regional or CONUS forecast model?

 

[Jeff Duda]

Despite the improvements in modeling, isn't there an upper limit to the accuracy that can be achieved simply because we cannot (currently) get actual observations with enough density or frequency?

Yes, but since we don't know how the spatiotemporal density of observations will change (or improve) in the future (mostly because widespread observations are based on funding which is a bureaucratic issue), we don't know 1) what that limit is, and 2) if at some point below that limit we will get to the point where highly accurate forecasts of CI can be made.

 

[rdale]

Is there something different about the longer-range climate and global temperature models that gives us a greater level of confidence than we seem to have with any other models?

Realize that climate models spend a lot of time modeling much more than the parts of the globe that impact tomorrow's storm locations. Here's a good write-up: http://andthentheresphysics.wordpress.com/2014/06/15/can-we-trust-climate-models/ "just because we can’t trust all aspects of today’s climate models doesn’t really mean that we have no reliable evidence for climate change or that we should simply decide to wait until we have climate models that are better"

 

[Stan Rose]

Is this a record number of consecutive posts from the same person on the same topic?

This thread reminds me of a question I have always had, and believe me it is not my intent to start any kind of argument or debate about climate change, which is obviously a hot button topic like religion or politics - this is truly offered as a serious question: Is there something different about the longer-range climate and global temperature models that gives us a greater level of confidence than we seem to have with any other models? We all distrust model output beyond 5 to 7 days, so wouldn't the extrapolation and compounding of biases and errors going out years and decades render those long-range climate models completely worthless? Or are those models simply a different animal, i.e., maybe they are large scale and global, and focused on fewer parameters, actually leaving less room for error as compared to a regional or CONUS forecast model?

That's a fair question. I agree with rdale, and to answer more directly: the main reason you can trust a climate model and not a forecast model is that weather is chaotic, and climate is not. Some aspects might be arguably chaotic, but to my knowledge no one has proven that climate is chaotic. I'd rather not go into an in depth discussion of non-linear dynamical systems (cause the last time I studied it was about 20,000 beers ago...) but basically it means that those small errors in observations/input that wreck havoc over time in a forecast model are not going to behave the same way that errors in a climate model will tend to behave as the system evolves over time. As you correctly figured, they are different animals.
There's a fundamental difference between weather and climate, which the naysayers ignore time and time again. It's a no-brainer for me to predict it will be colder here three months from now. I can tell you with great confidence that there will be lightning strikes in CO, tomorrow, 7 days from now (hugely confident!) and even 30 days from now! (reasonably confident!). I just can't tell you where the storms will hit. Likewise, I may not be able to tell you if the weekly temperature will be above or below average right at this spot 10 years from now, but if I'm satisfied with the assumptions, I can be confident on the global average trend.

 

[Royce Sheibal]

This has a little to deal with models and a lot to deal with forecasting. When I was doing my undergrad I did a massive study of the SPC's accuracy on predicting moderate / high severe weather events. During the spring and summer, the SPC scored a skill score of around 0.50, not bad. During the fall and winter months their skill score dropped below zero (worse than chance). Clearly the conclusion was the SPC's ensemble models sucked on fall and winter outbreaks, or that they were being analyzed wrong. Just goes to show that every model has is own strengths and weaknesses, including what time of year you are forecasting.

 

[Jeff Duda]

This has a little to deal with models and a lot to deal with forecasting. Clearly the conclusion was the SPC's ensemble models sucked on fall and winter outbreaks, or that they were being analyzed wrong.

Um...what...are you talking about?

SPC doesn't run any ensemble models. Rather they post-process severe weather related parameters from the SREF, which is a modeling system. Interpretation of the model output is generally not the problem for forecasters. A good forecaster knows how to read the output. Models indeed have their own biases and error distributions. Since you did a massive study, I imagine you produced a write-up/report on it. I would certainly like to read it.