gribify using ieee
Introduction
Writing grib-2 files can be quite painful with all the libraries
involved. However, there is a type of grib which is relatively easy
to write. In this format, a single grib2 record looks like
(binary header) variable number of bytes
(big endian ieee data) 4*ndata bytes
(binary trailer) 4 bytes, ascii '7777'
You can obtain the header/trailer using the -grib_ieee
option.
$ wgrib2 -order raw -d 1 in.grb2 -grib_ieee out
1:0:d=2008022101:APCP:surface:1 month fcst:
$ ls out*
out.grb out.h out.head out.tail
The first line takes the first record of the file and creates
out.grb ieee grib2 file (one record)
out.head binary file with the header
out.tail binary file with the trailer
out.h C header
Making a simple grib file
Continuing our example, we need some big endian binary data (ieee.bin).
We extract the ieee data (ieee.bin) using wgrib2. Normally you
would create the ieee data by another program.
$ wgrib2 -d 1 in.grb2 -no_header -ieee ieee.bin
1:0:d=2008022101:APCP:surface:1 month fcst:
Once we have the field in big-endian ieee, we can make
the grib file.
$ cat out.head ieee.bin out.tail > first.grb
$ wgrib2 first.grb -grib_out second.grb
1:0:d=2008022101:APCP:surface:1 month fcst:
The "cat" combines the header, data and trailer. The file,
first.grb is a valid grib2 if there were no undefined values
in "ieee.bin" (see the following section, "Undefined grid points").
The wgrib2 converts any undefined grid points to a bit
mask and uses a more common compression scheme.
Of course the example is pretty lame as it has the same date and
variable as the original file. However that can be changed.
$ wgrib2 second.grb -set_date 2008010100 -set_var SNOD -grib_out third.grb
1:0:d=2008010100:SNOD:surface:1 month fcst:
Making a grib file with C
The header file may looks like this,
unsigned char head[] = {
71, 82, 73, 66, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 164, 234, 0, 0, 0, 21,
1, 0, 7, 0, 1, 2, 1, 1, 7, 216, 4, 14, 0, 0, 0, 0, 1, 0, 0, 0,
72, 3, 0, 0, 0, 41, 16, 0, 0, 0, 0, 6, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 144, 0, 0, 0, 73, 0, 0, 0, 0, 0,
0, 0, 0, 5, 93, 74, 128, 0, 0, 0, 0, 48, 133, 93, 74, 128, 21, 79, 4, 96,
0, 38, 37, 160, 0, 38, 37, 160, 0, 0, 0, 0, 34, 4, 0, 0, 0, 0, 3, 5,
2, 0, 180, 0, 0, 0, 1, 0, 0, 0, 0, 100, 0, 0, 1, 134, 160, 255, 0, 0,
0, 0, 0, 0, 0, 0, 12, 5, 0, 0, 41, 16, 0, 4, 1, 0, 0, 0, 6, 6,
255, 0, 0, 164, 69, 7,};
unsigned char tail[4] = {55, 55, 55, 55};
#define NDATA 10512
#define SEC0 0
#define DISCIPLINE 6
#define EDITION 7
#define SEC1 16
#define CENTER 21
#define SUBCENTER 23
#define MASTERTABLE 25
#define LOCALTABLE 25
#define YEAR 28
#define MONTH 30
#define DAY 31
#define HOUR 32
#define MINUTE 33
#define SECOND 34
#define SEC3 37
#define SEC4 109
#define PRODUCTDEFTEMPLATENUM 116
#define PRODUCTDEFTEMPLATE 118
#define PRODUCTCATEGORY 118
#define PRODUCTNUMBER 118
#define SEC5 143
#define SEC6 155
#define SEC7 161
The header has definition of the header and trailer and
the location of selected elements in the header. Here
is an example that uses the above header (g2.6.h)
#include <stdio.h>
#include <stdlib.h>
#include "g2.5.h"
// Method #2 of writing a grib file
//
// sample program to create a grib2 ieee file
// to convert to jpeg file, type wgrib2 in_file -grib_out out_file
//
// can also use wgrib2 to change the variable name, level and time
// by the set_* commands
//
// 4/2008 Wesley Ebisuzaki
int main() {
FILE *grib;
float r;
char s[4], *t;
int i;
grib = fopen("test.grb", "wb");
// to change octet N of section 4 to the value of K
// add the following line:
// head[SEC4+(N-1)] = K;
// the grib documentation starts numbering octets at 1
// C uses 0 as the base
head[YEAR] = 2008 / 256;
head[YEAR+1] = 2008 % 256;
head[MONTH] = 2;
head[DAY] = 3;
head[HOUR] = 4;
head[DISCIPLINE] = 0;
head[PRODUCTCATEGORY] = 1;
head[PRODUCTNUMBER] = 2;
// write grib header
i = fwrite(head, 1, sizeof(head), grib);
printf("size of head = %d\n",i);
// for missing values, use the value NaN
// note: wgrib2 will convert 9.999e20 to missing
// write out IEEE big-endian data (NDATA)
// silly array -- needs to be in big-endian
r = 1.0;
t = (char *) &r;
// byte swap on a little endian machine
s[0] = t[3];
s[1] = t[2];
s[2] = t[1];
s[3] = t[0];
for (i = 0; i < NDATA; i++) {
fwrite(s,1,4,grib);
}
// write grib trailer
fwrite(tail,1,sizeof(tail),grib);
return 0;
}
As with the first example, one should convert undefined grid points and the ieee packing
to a more commonly used packing using "wgrib2 in.grb -grib_out out.grb".
In this example, in.grb2 happens to have the correct grid
but wrong the center. If you examine in.grb2, you see that it comes from
NCEP and you happen to be working in in Kiribati (196 in WMO table). That can be easily fixed.
$ wgrib2 in.grb2 -center
1:0:center=US National Weather Service - NCEP (WMC)
$ wgrib2 in.grb2 -set_center 196 -grib_out third.grb2
1:0:d=2008022101:APCP:surface:1 month fcst:
$ wgrib2 third.grb2 -center
1:0:center=Kiribati (NMC)
Are all the necessary -set_* options available?
Probably not as this this procedure for making grib file is
a work in progress. Send comments to wesley.ebisuzaki@noaa.gov.
Getting the "template" grib2 file
The procedure depends on having a grib2 file with the
correct grid. For common grids, you can often find a
such a grib file. Sometimes you can find a grib1 file
and then you can use
cnvgrib to convert the file into grib2. As a last
resort, one could take a grib1 file, convert it to
the correct domain by the program copygb and then use
cnvgrib.
Of course, the last resort is the hardest because it requires
knowing the grid description in the arcane "GDS" format.
(You'll have to read the fortran code and know a bit about grib1.)
5/2010: wgrib2 can make regular lat-lon template files
by the -lola option.
Undefined grid points
In grib, undefined grid points are usually specified by a
bit mask. However, ieee floating point numbers can have a
value of NaN or "not a number". The current WMO documentation
doesn't specify how to interpret a NaN, so wgrib2 interprets
a NaN as undefined. Seems logical, doesn't? Anyways I'll assume
logic holds until WMO defines otherwise. However,
writing a NaN can be difficult in some languages. So
one can use a special number as the undefined value. If
you use 9.999e20, you can fix the file by
wgrib2 in.grb2 -grib_out out.grb2
If for some reason, you decided to use -999 as the undefined value,
you can fix the file by
wgrib2 in.grb2 -undefine_val -999 -grib_out out.grb2
Other grib decoders
The ieee variation of grib2 is not supported by the NCEP fortran and C
libraries and programs based on those libraries. You can convert to
a more standard packing by using wgrib2.
wgrib2 in.grb2 -grib_out out.grb2
Warning
Please check the results of gribifying your data. This
facility is very new and variations of grib files is many.
Please plot the results.
|