Python and Weather

[Kelton Halbert]

I didn't feel like this was better suited for the equipment forum, but if it needs to be moved, my apologies!

I wanted to discuss the use of Python in weather data analysis. This is something I have picked up lately and wanted to see if there was anyone else out there using Python to do anything cool with weather data.

So... anyone out there?


I'll start by sharing what I've been doing though.

I'm using matplotlib and the Enthought Python Distribution (EPD) to plot and contour raw METAR surface data gathered from http://weather.noaa.gov/pub/data/observations/metar/cycles/
I had noticed most surface analyses are from model runs like the RUC, but I wanted to see what would happen with the raw METAR data. It's working quite nicely! Here's a little example...

There are some more details on whats going on in this blog post I made. http://tempestchasing.com/10/11/2011/other/weather-and-python-a-new-frontier/

Your turn to share! I'm very curious to see what else is going on out there.

Kelton.

 

[M Conder]

Great work. As a old GEMPAK vet I agree with your blog comments about it, and am looking forward to seeing what you can do with Python.

 

[Kelton Halbert]

Great work. As a old GEMPAK vet I agree with your blog comments about it, and am looking forward to seeing what you can do with Python.

Thank you!

Although, my intention wasn't to self promote here... I was hoping to spark some dialogue with other people doing similar or even not so similar things with Python. I know of a few members here that use Python as well and thought maybe there would be more. I guess not though!

 

[Patrick Marsh]

I'm a big Python guy. In addition to rewriting NSHARP in Python (see my talk at the AMS Python Symposium in January!) I also make weather maps. Here's something I did for the OU Synoptic class regarding the Alaskan bomb last week.

http://weather.ou.edu/~metr4424/archive.php?chart=early_nov_alaska&numhours=120&interval=6

 

[Kelton Halbert]

I'm a big Python guy. In addition to rewriting NSHARP in Python (see my talk at the AMS Python Symposium in January!) I also make weather maps. Here's something I did for the OU Synoptic class regarding the Alaskan bomb last week.

http://weather.ou.edu/~metr4424/archive.php?chart=early_nov_alaska&numhours=120&interval=6

I will have to stop by when I go to AMS then!

I think that may be the coolest thing I have seen yet. What levels are you using for the winds? I'm assuming 500mb or higher? Great map.

Were you also using matplotlib/basemap? If you were, I'd like to know how you got the color bar padding down so low... I can't seem to figure out how to get rid of the space between the plot and the bar.

I am constantly amazed by how versatile python is in its capabilities, and even more so how easy it is to do it. I have had no real previous coding experience and found it easy to pick up!

 

[Patrick Marsh]

Kelton, everything on that map is computed at what is known as the Dynamic Tropopause. For this purpose, I've defined the tropopause to be the 1.5 PVU (Potential Vorticity Unit). I then took the temperature and pressure and converted it to potential temperature which is a conserved quantity. The general idea here is that the warmer colors indicate a higher tropopause which would be indicative of a warm, and potentially moist, troposphere. The cooler colors indicate a lower tropopause which would indicate a colder, drier troposphere. The dynamic tropopause is a great place to look for large scale troughs and ridges because 1.) Potential temperature is conserved and 2.) everything is advected around by the wind on the tropopause.

As for how I changed the colorbar, I used the ImageGrid functionality. One of the keyword arguments is cbar_pad, which is a measure of the spacing between the image itself and the colorbar. Additionally, ImageGrid is a more robust way of doing paneling, which I've come to do quite frequently.

To import ImageGrid you use the following import statement

from mpl_toolkits.axes_grid1 import ImageGrid

 

[Tarmo Tanilsoo]

I got hooked into Python when I had a programming course in the university. As far as I go, I have used Python for example to decode Level III radar images.

 

[Kelton Halbert]

Kelton, everything on that map is computed at what is known as the Dynamic Tropopause. For this purpose, I've defined the tropopause to be the 1.5 PVU (Potential Vorticity Unit). I then took the temperature and pressure and converted it to potential temperature which is a conserved quantity. The general idea here is that the warmer colors indicate a higher tropopause which would be indicative of a warm, and potentially moist, troposphere. The cooler colors indicate a lower tropopause which would indicate a colder, drier troposphere. The dynamic tropopause is a great place to look for large scale troughs and ridges because 1.) Potential temperature is conserved and 2.) everything is advected around by the wind on the tropopause.

As for how I changed the colorbar, I used the ImageGrid functionality. One of the keyword arguments is cbar_pad, which is a measure of the spacing between the image itself and the colorbar. Additionally, ImageGrid is a more robust way of doing paneling, which I've come to do quite frequently.

To import ImageGrid you use the following import statement

Wow, that's some awesome stuff. I'll have to remember that...
Thanks for the point in the right direction with ImageGrid. I'll have to dig around and play with that. And speaking of conserved quantities, I've just added Theta to my surface map (along with several other things) and have plans to implement theta-e and mixing ratio computed from the METAR data. I think the equations have some approximations, but for the most part it is working. I'm only in my Jr. year of high school (Algebra2) and have no clue what I am dealing with when it comes to Calculus, so my friend at OU gave me some equations I could work with.

Tarmo! I remember talking to your about this over twitter. Thats a pretty cool program. In fact, I'd love to see an example, if at all possible.
Were you able to decode all types of data? Like velocity and dbZ? Also, have you thought about working with level II data from the IA State server?

 

[Tarmo Tanilsoo]

Yeah, this is one early example developed in Python/Tkinter/PIL.

It shows only reflectivity[note, the time is in my local time, 8 hours ahead of Central]. I am intending to recode the stuff in the future...

As far as Level 2 data goes, I haven't tried. If anyone could point me to its documentation, I might give it a try in a further future.

 

[Kelton Halbert]

Sorry for the late reply, I was out of town for Thanksgiving.

That looks awesome! I'd certainly use that program any day.

Level-II docs? Sure thing! That's what the idea behind this thread is... sharing ideas and tips. Let me see if I can find anything...

I'm honestly not sure where to start, and my battery is about to die, but I'd recommend searching through this website: http://www.roc.noaa.gov/WSR88D/Level_II/Level2Info.aspx

Good luck!

 

[Kelton Halbert]

Not sure if anyone else is reading this or cares, but in the spirit of sharing information, I'd thought I'd post a few things here...

I was wanting to plot surface vorticity and divergence based on interpolated wind grids (computed from the raw metar stations), but ran into an issue where I needed the distance between each grid point in meters. Thanks to Patrick, I'm now using a 40km RUC grid, which has allowed for me to plot vorticity and divergence using numpy.gradient. I'd recommend to anyone else in the future that wants to plot gradients of any sort to avoid operating a grid on degrees of lon/lat, you'll end up having to scrap it anyway. Vorticity is working fantastically - divergence is a little noisy, but works none-the-less!
Here's an example of surface vorticity computed thanks to this new grid. Units are multiplied by 10 to the 5th. Ignore the image filedate, that was when the image was generated. The data is from December 5th at 18z.

More information on the other cool bits of progress made can be read on my blog.

 

[Kelton Halbert]

Here's a general question for users of Python: Has anyone else been having issues with the numpy.gradient function? I've been using it to asses the advection of scalar quantities like temperature and mixing ratio, but have been running into issues. The issues have been solved thanks to Greg Blumberg's great troubleshooting, but figured this may be something important to share.

Using numpy.gradient to asses temperature advection:
A temperature gradient oriented from north to south returns proper values. The error comes from temperature gradients oriented from west to east. For some reason, the numpy.gradient function is drawing the gradient vector from highest value to lowest value, when the vector is supposed to point from lowest value to highest value. This is solved in the advection equation by tacking on a -1: wind_vector * del Temperature = u_component * dTemperature/dx + v_component * -1 * dTemperature/dy

Does anyone know if numpy.gradient is purposefully built to do this? The dx portion of the equation does not have the same issue.

 

[Patrick Marsh]

I honestly haven't had time to think about it, but my knee-jerk reaction would be to investigate to see if this is an artifact of how meteorology describes its coordinates. In meteorology 0 is N, 90 is E, 180 is S, and 270 is W. In mathematics, 0 is E, 90 is N, 180 is W, and 270 is S. I'd suggest looking there.

 

[Kelton Halbert]

Sorry for the late reply Patrick.

I've asked Greg if this is the case, as he was the one that tested this, and he says that this does not appear to be the case. Perhaps he is better suited to explain what he did, but from what he told me, he used random/unitless numbers on either a 3x3 or 4x4 matrix and came up with the results that pose this question.

 

[Jeff Duda]

divergence is a little noisy, but works none-the-less!

This post isn't really related to the main topic of this thread, but this is a good place to throw this in:
What you may think is "noise" in the divergence field is likely signal that the surface stations are poorly resolving. Check out the surface divergence field from a 3 km WRF simulation in this image. And this is just at 3 km grid spacing. If you went down to < 1 km, you'd see even more detail/"noise". So don't be discouraged if your plots don't turn out the way you expected. It may be legitimate!