|
Fast Downloading of Grib, Part 2
|
Wrappers @ NCDC
|
|
While the procedure detailed in part 1 is straight forward, it could be
easier. I don't like looking for and typing out URLs. Writing loops takes time too. Less experienced
people like it even less. Dan Swank wrote a nice wrapper to download data
for the North American Regional Reanalysis (NARR). It worked so well that he did
get-httpsubset.pl.
During May 2006, 95% of the NCDC-NOMADS downloads were done using cURL.
|
|
Wrappers @ NCEP (NOMADS): get_gfs.pl
|
|
At NCEP, we wanted people to (1) get forecasts using partial-http transfers
rather than ftp2u and (2) move off the nomad servers to the more reliable
NCO servers. So get_gfs.pl was born. Wanted the script to be easy to use,
easy to reconfigure, easier to install and work with Windows.
|
|
Requirements
|
- get_gfs.pl.
- perl
- cURL
|
|
Configuration
|
- The cURL executable needs to be downloaded and put in a directory on your $PATH.
- The first line of get_gfs.pl should point to the location of the local perl interpreter.
- Non-windows users can set the $windows flag to "thankfully no" in get_gfs.pl for more efficiency.
|
|
Usage:
|
get_gfs.pl data DATE HR0 HR1 DHR VARS LEVS DIRECTORY
Note: some Windows setups will need to type:
perl get_gfs.pl data DATE HR0 HR1 DHR DIRECTORY
DATE = start time of the forecast YYYYMMDDHH
note: HH should be 00 06 12 or 18
HR0 = first forecast hour wanted
HR1 = last forecast hour wanted
DHR = forecast hour increment (forecast every 3, 6, 12, or 24 hours)
VARS = list of variables or "all"
ex. HGT:TMP:OZONE
ex. all
LEVS = list of levels, blanks replaced by an underscore, or "all"
ex. 500_mb:200_mb:sfc
ex. all
DIRECTORY = directory in which to put the output
example: perl get_gfs.pl data 2006101800 0 12 6 UGRD:VGRD 200_mb .
example: perl get_gfs.pl data 2006101800 0 12 12 all sfc .
|
|
Advanced Users
|
|
A user asked if it were possible to mix the variables and levels.
For example, TMP @ 500 mb, HGT @ (250 and 700 mb). Of course you could
run get-gfs.pl twice but that wouldn't be efficient.
It is possible because get-gfs.pl uses regular expressions and
regular expressions are very powerful. All you need to remember
is that get-gfs.pl converts the colon and underscore to a vertical
bar and space, respectively for the VAR/LEV arguments.
Unix/Linux:
get-gfs.pl data 2006111500 0 12 12 all 'TMP.500 mb|HGT.(200 mb|700 mb)' data_dir
Windows:
get-gfs.pl data 2006111500 0 12 12 all "TMP.500 mb|HGT.(200 mb|700 mb)" C:\unix\
|
|
Wrappers @ NCEP (NCO): get_data.sh
|
|
NCO (NCEP Centeral Operations) also has a wrapper,
get_data.sh.
|
|
Created: 10/2006
comments: Wesley.Ebisuzaki@noaa.gov
|