GFS grib2 data

[MClarkson]

So I have a full grib2 file, containing about 300 fields. Its about 55 megs. I need to grab a few values at a certain point. Using the wgrib2 -lon function to select location and the grep command to narrow down to the fields I want works, in that it does return the values I require. But it is so slow! Smaller files speed things up, but then the phase where I use wgrib2 and grep to go from big grib file to little grib file is still slow.

Anyone think there is a faster way?

Thanks

 

[Jeff Duda]

Not to my knowledge. I'd ask Wesley Ebisuzaki (the NOAA GRIB guru). I'm doing a lot with GRIB2 GFS data as well, so I know what you're talking about. I use Grads if I need a specific value, but only in the context of running a script.

 

[rdale]

Have you tried some of the GEMPAK point programs?

 

[Tim Supinie]

Python/PyNIO would be excellent for this, but it can be a bit of a learning curve, especially if you haven't used Python at all before. And PyNIO can be painful to set up on some systems.

 

[Jeff Duda]

I was asked to post this to the forum on behalf of Wesley Ebisuzaki:

"So I have a full grib2 file, containing about 300 fields. Its about 55 megs. I need to grab a
few values at a certain point. Using the wgrib2 -lon function to select location and the grep
command to narrow down to the fields I want works, in that it does return the values I require.
But it is so slow! .. Anyone think there is a faster way?"

Most NCEP grib2 files are compressed with JPEG2000 which produces small but slow-to-decode
files. The best trick is to reduce the number of decodes that you do.

Simple way to get values at (30N, -90W) and (40N, -80N)

wgrib2 IN.grb -lon -90 30 > location1.txt
wgrib2 IN.grb -lon -80 40 > location2.txt

Trick 1, extract location1 and location2 at the same time (2x savings)
Code that reads location.txt needs to be changed

wgrib2 IN.grb -lon -90 30 -lon -80 40 > locations.txt

Trick 2, only decode the fields that you need (ex. 2 m temp, accumulated precip)
2 m temp = "TMP:2 m above ground"
accumulated precip = "APCP"

wgrib2 IN.grb -match "TMP:2 m above ground:|:APCP" -lon -90 30 -lon -80 40 > locations.txt

notes: -match "(A|B)" means only process fields whose inventory matches A or B.
The argument of -match is a regular expression (unix speak) which is expressive.

In our example, only two fields need to be decoded as compared to the 2x300 fields.

 

[MClarkson]

That -match command might be exactly what I was looking for. Initial testing makes it look faster than grep.

Thanks!

Oh ya, that screams through big grib files way faster. Nice!

Tell Wesley I said thanks!