GEMPAK Scripting

[Kelton Halbert]

EDIT: DOWNLOAD LINK BELOW
Greetings everyone! Let me start by introducing myself, I am new here at StormTrack.

My name is Kelton Halbert, and I am a high school student that lives in Nashville, Tennessee. I've been chasing storms for about two years, and this year was my first trip out to the Great Plains. Unfortunately that trip was the May 11th bust. Oh well, it leaves me with more to look forward to...

But enough about me. This thread is about GEMPAK Scripting.

I stumbled upon GEMPAK earlier this year, and realized what a great tool it was. I also realized the issue with pulling data. Further research (including some threads in here) led me to realize that I was going to have to write my own scripts to do the job.

I'm writing this thread because I want to know - would ANYONE be interested in having these scripts? I'm all about open source (free stuff!), and the code is original. Right now it is written in UNIX (I'm running GEMPAK on my Mac) but converting it to LINUX won't be a problem at all. Also, these scripts are in bash, not csh.

So, is anyone interested?

Right now I have a script that I've tried to make as easy to customize as possible for polling model data. Right now, it is set up to download the hourly RUC2 data from the NOMADS NCEP HTTP server, but with a little configuration, it can work with the NCEP FTP and NWS FTP servers, and any model data of choice. Within the script are the commands to decode the .grib2 files (assuming you already have GEMPAK installed) and remove the .grib2 files after they have been decoded. Lastly, it is set up to delete the previous model run (configurable in units of hours). There is a similar script that does the same thing for satellite imagery. Doing it this way, however, requires some small editing to your datatype.tbl file for NMAP2 to recognize the data.

This and several other scripts I have written are meant to run from crontab at set intervals. Unfortunately, timing is everything, so it may take several tries before the script is configured correctly. However, I am willing to help configure and troubleshoot when I have time available.

I understand that you can buy GEMPAK scripts, and in no way is this meant to be competition. The fact is, the paid scripts are probably cleaner, more efficient, and do more. However, I figured if I have already done this much work, I should at least share it in the event someone else needs it.

I'll be finishing up some last minute touches and cleaning up some of the code over the next few days. If there is interest, I'll have a download URL up and a short tutorial on how to get started.

If anyone would like a sample, this is the end result.

http://tempestchasing.com/models/ruc/

EDIT: DOWNLOAD LINK
http://tempestchasing.com/gempak/

 

[rdale]

Good work! It's a great feeling to start from pretty much scratch and pull off this feat...

 

[Kelton Halbert]

Thanks rdale! It is a good feeling.

I'm running a little behind on getting things on the internet because I am out of town. However, the scripts themselves have been touched up and a few more bugs have been worked out.

I'll try to have them online soon.

 

[Ben Holcomb]

I'd say just post them on a website. People will find them and use them I would imagine. Check out http://www.simuawips.com if you want to see what you can truly do with GEMPAK scripting - My buddy Andrew has customized the heck out of it, including colors and what not and added it to a nice little javascript/ajax interface.

 

[Kelton Halbert]

Ben, I love SimuAwips. Props to your friend!

Here's the awaited download link: http://tempestchasing.com/gempak/
Most of the information you need to know is on that page. Also, included in the .zip download, there is an installation instruction file. However, just a few notes...

1) I only have the UNIX scripts up. LINUX users will have to wait a little while longer.

2) You MUST have WGET installed. Google how to do that!

3) The included crontab file is for Central Daylight Time only. The values need to be changed when converting from CDT to CST, or any other time zone. The only way to get around this is to run your system clock on UTC, in which case, the values would need to be converted to UTC.

If there are any questions or issues, let me know!

 

[Kelton Halbert]

Alrighty,

Give me a few days and I'll have a new update. This will include:

1) A script to download Level-II radar data
2) A script to download GFS 0.5 DEG model data
3) An updated crontab with better time scheming.

The scripts are already done, I just need to optimize them and then upload the update. I'll post when its done!

P.S., I haven't had time to check LINUX compatibility with these scripts. If someone would like to test, they are more than welcome too.

 

[Kelton Halbert]

Scripts are now updated with the Level2 data. The download link can be found at http://tempestchasing.com/gempak/

There are two scripts for this. I set the first to run on crontab every 5-10 minutes, but I would encounter an error where data 10 minutes before the hour would be missing. So, the offset script is meant to fix this. I'll be honest, I cannot remember what I set it to run at with crontab (it's been months since I've used this regularly), so play around with it. If anyone has trouble, let me know and I'll work on it.

Edit: GFS 0.5 DEG data script is, in fact, included.

 

[Kelton Halbert]

I know people are wondering, "Why on earth does this guy keep posting?" Well, I've got news for you! New version!

Upon request of my good friend, Greg Blumberg, this update includes scripts to download surface METAR data, 6 minute profilers, hourly profilers, and upper air data.

Scripts Added:
- soundings.sh
-Downloads national upper air sounding data. Can also be loaded into NSHARP, the sounding viewer
- profiler_download.sh
-Downloads hourly profiler data
- metars.sh
-Downloads surface METAR data
- 6min_profiler_download.sh
-Downloads 6 minute profiler data

Data is coming from http://metfs1.agron.iastate.edu/data/
Unfortunately I couldn't find a way to decode .upa and .sao files, so it downloads pre-generated GEMPAK files form the server. The down side is that the files are based on day and not by hour, so the script re-downloads the file. The good news is that WGET will NOT re-download the file unless it has been modified since the last download. If anyone knows of a better way to get profiler/upper air/surface data and decode it, I am all ears!

As always, the download link is http://tempestchasing.com/gempak/