Introduction

wgrib2 is not simple upgrade of wgrib for grib-2 files. wgrib can slice and dice grib-1 files. wgrib2 is more like four drawers of kitchen utensils as well as the microwave and blender. This kitchen miracle was done by a more developer friendly design and the contributions of many people.

• inventory and read grib-2 files (grib-1 is not supported)
• manipulate (reorder, extract) grib-2 records (messages)
• ieee, text, binary, comma separated values (CSV) output
• extract data from specified latitude/longitudes
• converstion to netcdf (compile time option)
• export to a mysql database (compile time option)
• script-level parallel programs
• regional subsets
• creating new grib-2 files

Converting from Grib-1 to Grib-2

GRIB-2 has arrived and has begun replacing the the older standard GRIB (GRIB-1). Since GRIB-2 can reduce the file size by upto 80% (NAM, JPEG2000 compression), converting to the new standard may be faster than some expected.

Sometimes you want to use a feature of wgrib2 on a grib-1 file. Since wgrib2 doesn't do grib-1, you are out of luck unless you convert the file into grib-2. Some files can be converted into grib2 by the cnvgrib utility.

News

2/2007: Manfred Schwarb went over the code last Christmas-New Years and sent in 31 patches. 31 patches sounds like a lot but he must have used a spell checker on the code. Anyways some of the patches were for the more unpolished code in the package which caused me to go over that code. Manfred also sent in a script that can read a grib2 table from the web documentation and create C code. Much better than typing by hand or any other script that I had in my tool box.

9/16/2005 alpha version is released to the public.

Some Good Stuff

• Almost trivial to add custom features!
• Customized inventories
• Table driven
• Code is reorganized
• Super easy integration of new and local options
• wgrib2 knows the lat-lon of the individual grid points for several grids
• Designed for multiple developers (hint, hint, hint)
• Works with the partial-http downloading aka "Fast Downloading of GRIB Files"

Bad Stuff

• Uses external libraries (png,jpeg,netcdf,compression,grib2 decode) which can make compiling and linking a real pain. At least all the external libraries are in C.
• interpolation to lat-lon values is done by nearest neighbor.
• wind rotation is not handled

Usage

• similar to wgrib but different
• each option call corresponds to a subroutine call
• type wgrib2 to see primary options
• inventory format is specified on command line by options
• if no "inv" option is specified, -s is used

Joining the development effort

• Source code: knowledge of C and grib-2
• Donated source code / non-trivial bug fixes must either be GNU or public domain.
• Prefer PD bug fixes for PD code and GNU bug fixes for GNU code.
• Bug reports are important
• Contact wesley.ebisuzaki@noaa.gov

Contributions by

• Wesley Ebisuzai: many modules
• Reinoud Bokhorst: tosubmsg, checksum
• Jaakko Hyvätti: gribtab
• Kristian Nilssen: netcdf module
• Karl Pfeiffer: georeference
• Pablo Romero: unix_time
• Manfred Schwarb: many modules
• Arlindo da Silva: openGrADS, bbox
• Niklas Sondell: mysql module
• Sergey Varlamov: netcdf module improvements, georeferencing updates
• thanks to the people who report the bugs and more who provide the fixes!

Documentation

• converting from wgrib to wgrib2 in scripts
• submessages
• writing a simple function
• selecting fields to decode
• option types: init inv out misc
• bin, ieee, text format
• small fast databases
• usage questions
• compile questions
• some tricks

An incomplete list of the options

• -AAIG export arcinfo ascii grid
• -append set append mode flag
• -big_endian set mode to big-endian ieee
• -bin write out file in binary format
• -bitmap prints details of bitmap
• -center prints details of center that wrote the file
• -checksum checksum of various sections or message
• -code_table prints code table I.J
• -config prints version and compile options
• -count prints the number of fields
• -d dump specific record
• -disc prints the discpline
• -end end/stop/bye/quit after 1 line of inventory
• -ens print ensemble information
• -fi closes -if
• -fix_ncep convert cnvgrib time codes to standard codes
• -flush flush output buffers after every write
• -for select a range of records to process
• -for_n select a range of records to process
• -ftime prints the forecast time
• -g2clib select internal vs g2clib decoding routines
• -get_byte show selected bytes of the grib message
• -GRIB write current grib message (potentially many fields)
• -grib write current grib submessage (one field)
• -grib_out writes the current field in grib format
• -grid prints out the grid information
• -grid_changes prints the number times the grid changed
• -h shows a list of common options
• -header write f77 or text header
• -help shows help for X
• -i reads inventory from stdin for record selection
• -ieee write data in IEEE format
• -if if this then that
• -ij print value at grid point X, Y
• -ijbox print values at grid point (ix1:ix2, iy1:iy2)
• -ijlat print lat, lon, value at grid point X Y
• -ijsmall_grib write grib file for subdomain
• -ilat print lat, lon, value of Nth grid point
• -ilat redirects inventory to a file
• -lev prints the level
• -little_endian sets mode for little-endian ieee
• -lola
• -lon prints the lat, and value of the grid point nearest the specified lat-lon
• macros: -s -verf -V
• -match process records that match a posix extended regular expression
• -match_inv the inventory used by -match
• -quit stops processing
• -max prints the maximum value
• -min prints the minimum value
• -MM prints the month as two digits
• -prob prints the probability field if any
• -n print inventory line number
• -ncep_norm normalize average/accumulation periods
• -no_header do not write f77/text header
• -not process records not matching a regular expression
• -if if not this then that
• -npts prints number of points of grid
• -nxny prints nx ny of grid
• -one_line puts inventory record on one line
• -order prints the scan order of the data
• -rpn Reverse Polish Notation calculator
• -s print a simple inventory
• -set_byte set selected bytes of the grib message
• -set_grib_type set grib packing (not for -grib)
• -set_metadata change the metadata, X=input metadata file
• -set_var change the field name
• -small_grib write regional subset
• -stats prints some statistics about the fields
• -t reference time
• -T reference time with seconds
• -tigge use tigge-style names
• -tosubmsg combine messages into one message man many submessages
• -undefine undefine/maskout grid points
• -unix_time print unix time (seconds since 1/1/1970)
• -v0, -v, -v2 verbosity level
• -vt verification time
• -VT verification time with seconds
• -verf simple inventory using forecast verification time
• -version version of code
• -V prints a verbose inventory
• -wind_speed calculate the wind speed

Exporting data to other programs

• -netcdf: write data in netcdf format
• -mysql: export data to a mysql datdabase
• -mysql_speed: export data to a mysql datdabase
• -spread: write data for spreadsheets
• -csv: write in column separated values, another one for spreadsheets
• -text: data in text format
• -bin: data native binary floating point
• -ieee: data in big endian IEEE format
• -ijbox: write a rectangular grid of data
• -AAIG: arcinfo ascii grid, GIS

Specialized options used by other programs

• -ctl_inv: print parameters needed by g2ctl
• -ctl_ens: print parameters needed by g2ctl
• -domain: print out the lat-lon domain
• -cyclic: needed by g2ctl
• -lev0: print level information needed by g2ctl
• -limit: web programs that need controls of cpu usage

Seeing the details of the grib file

• -code_table_I.J: to see contents of code table I.J
• -flag_table_I.J: to see contents of flag table I.J
• -get_byte show selected bytes of the grib message
• -SecI: see descriptions about section I, I = 0..6,
• -Sec_len: see length of each section
• -0xSec I see hex dump of a section I

Writing grib2

1. Using the ieee packing option allows you to make some very light weight C programs that write grib2. You can even use standard Unix utilities. See -grib_ieee.
2. The basic -import_ieee/bin/text option.
3. Example of converting HDF to grib2
4. use -set_metadata

Machines able to run wgrib2

• Redhat linux x86_32, Ubuntu: install gcc, libpng12-dev
• Mandriva linux, needs png library installed
• AIX, use makefile_all_libs, some fiddling is needed
• Solaris, needs gnu make and gcc (old report)
• HPUX, needs changes to makefiles
• XP, used cygwin and had to download many libraries modified makefiles use builtin-jasper library.
• Intel-based Mac

System dependencies: Windows and others

The same source code is used for windows, unix, linux, 32-bit, 64-bit, big endian and little endian executables. The main difference between the windows and linux versions is that windows treats binary and text files differently. Consequently you can send a grib or binary data to STDOUT in unix/linux with no problems. Under windows, the results are implementation dependent.

32-bit executables are limited to 2 GB files. (The fseek function is limited to signed long integers.)

The maximum number of open files is system dependent.

The netcdf conversion is done in memory, so the speed is dependent on the amount of available physical memory.

Source Code and Compling Hints

The wgrib2 code is written to the POSIX standard. The regular expression package requires the regcomp and regexec routines which are POSIX-2. (This package is optional for systems which don't include these routines.) The wgrib2 code can be compiled with 32 or 64 bit pointers and integers. However, the code has to be compiled in a like manner to all the libraries. (You can't mix 32 and 64 bit libraries and wgrib2 code.) Some packages are optional (netcdf, mysql) you shouldn't enable them if not needed.

• FTP directory
• README
• Changes
• compile questions
• wgrib2.tgz
• Windows XP

Precompiled code from External Sites

There are sites with precompiled versions of wgrib2. This list is neither exhaustive nor an endorsement of the sites. I have not tested the wgrib2 executables from these sites and YMMV.

• OpenGrads: Darwin, Freebsd, Linux, Windows(cygwin)
• Fedora
• RPMs: Centros, Fedora, OpenSUSE, RedHat, SUSE

Status

The code is being developed on a linux x86_32 platform in standard (POSIX) C. Code is run on the AIX (64-bits) in NCEP operations.

Change logfile

Solutions #1

Higher resolution files may need a bigger stack.

sh-3.00$ wgrib2 rtgssthr_grb_0.083_awips.grib2 -V
Segmentation fault

Oh, oh. Increasing the stack size from 10MB to 20 MB does the trick. (The Jasper library wants a big stack when decoding a field with 9 million grid points.)

-sh-3.00$ ulimit -s 30000
-sh-3.00$ wgrib2 rtgssthr_grb_0.083_awips.grib2 -V
1:0:vt=2009062900:surface:anl:TMP Temperature [K]:
    ndata=9331200:undef=3246169:mean=287.003:min=271.33:max=312.8
    grid_template=0:
        lat-lon grid:(4320 x 2160) units 1e-06 input WE:NS output WE:SN res 48
        lat 89.958000 to -89.958000 by 0.083000
        lon 0.042000 to 359.958000 by 0.083000 #points=9331200

Bugs

Please report then to wesley.ebisuzaki@noaa.gov. When you report bugs, try to make them reproduceable on a linux machine and include sample data.

Distribution

The source code modules for wgrib2 are in the public domain or under the GNU licence depending on the authors of the various modules. Wgrib2 uses libraries that are in the public domain, under various GNU licences, the Image Power JPEG-2000 Public Licence, libpng licence, the zlib licence, the netcdf licence and perhaps others. How about one licence to rule them all?