Skip Navigation Links www.nws.noaa.gov 
NOAA logo - Click to go to the NOAA home page National Weather Service   NWS logo - Click to go to the NWS home page
Climate Prediction Center
 
 

 
About Us
   Our Mission
   Who We Are

Contact Us
   CPC Information
   CPC Web Team

 
HOME > Monitoring_and_Data > Oceanic and Atmospheric Data > Reanalysis: Atmospheric Data > wgrib2 questions
 

Compiling wgrib2 v3.0.0+

1) Download ftp://ftp.cpc.ncep.noaa.gov/wd51we/wgrib2/wgrib2.tgz
2) remove pre-existing grib2 directory if exists: rm -r grib2
3) untar wgrib2.tgz:  tar -xzvf wgrib2.tgz   (use gnu tar)
4) cd to main directory:  cd grib2
5) for wgrib2 v3.0.0 only,
     Update makefile 
       $ curl -o makefile "https://ftp.cpc.ncep.noaa.gov/wd51we/wgrib2/makefile_v2_for_wgrib2_v3.0.0"

The makefile uses two and one optional environment variables that have to be set.
  With wgrib2 v3.0.0, you need to set $CC and $FC, and the makefile tries to identify
    $COMP_SYS from "uname -a".  However, if the makefile cannot identify your system,
    $COMP_SYS will not be set.  If your compilers are gcc and gfortran, you can
    try compiling wgrib2 with COMP_SYS=gnu_linux.


  linux, bsd-type OS, gcc/gfortran compilers:  COMP_SYS=gnu_linux
                                               CC=gcc
                                               FC=gfortran
  linux, AOCC:                                 COMP_SYS=clang_linux
                                               CC=clang
                                               FC=flang
  linux, icc and ifort:                        COMP_SYS=intel_linux
                                               CC=icc
                                               FC=ifort
  linux, Nvidia HPC SDK                        COMP_SYS=nvidia_linux
                                               CC=nvfortran
                                               FC=ifort

  Windows, cygwin gcc and gfortran:            need a new version of grib2/makefile
                                               https://ftp.cpc.ncep.noaa.gov/wd51we/wgrib2/makefile_v2_for_wgrib2_v3.0.0
                                               COMP_SYS=cygwin_win
                                               CC=gcc
                                               FC=gfortran

  MacOS, real gcc and gfortran:                COMP_SYS=gnu_mac
                                               CC=gcc
                                               FC=gfortran



  not recently tested AIX, gnu_linux_g95, open64
  If COMP_SYS is not defined, the makefile will attempt to determine the COMP_SYS.

6) set environment variables and make, example

   export CC=gcc                    (bash version)
   export FC=gfortran
   export COMP_SYS=gnu_linux        (optional)
   make                             (to make wgrib2)
   make lib                         (to make wgrib2 library)

Compiling wgrib2 v2.0.6-v2.0.8

Compiling wgrib2 is easy on a linux system with gcc/gfortran or Windows with the cygwin compilers. Installing gcc and gfortran will depend on the linux distribution or OS.
1) Download ftp://ftp.cpc.ncep.noaa.gov/wd51we/wgrib2/wgrib2.tgz
2) remove pre-existing grib2 directory if exists: rm -r grib2
3) untar wgrib2.tgz:  tar -xzvf wgrib2.tgz   (use gnu tar)
4) cd to main directory:  cd grib2
5) Update makefile for gnu make version 4.3+ incompatability, needed for wgrib2 upto v3.0.0
    To check the version of make, 
       $ make --version
    for version wgrib2 v3.0.0, and gnu make v4.3+, get a new makefile
       $ curl -o makefile "https://ftp.cpc.ncep.noaa.gov/wd51we/wgrib2/makefile_v2_for_wgrib2_v3.0.0"
    for earlier versions of wgrib2 and gnu make v4.3
       $ sed 's/\\#/#/' makefile.tmp
       $ mv makefile.tmp makefile
6) define the C and fortran compilers and make
   Bash:
     export CC=gcc
     export FC=gfortran
     make
     make lib                        only if you want the ftn_api
   Csh
     setenv CC gcc
     setenv FC gfortran
     make
     make lib                        only if you want the ftn_api
7) See if wgrib2 was compiled
     wgrib2/wgrib2 -config

note: you may have to install gcc, gfortran, m4, and g++

Intel Compilers

Compiling wgrib2 with the intel compilers (icc and ifort) is supported with wgrib2 v2.0.6+. Gcc has to be installed to compile the Jasper library for jpeg2000 support. Note that icc and ifort may require some money. Compiling wgrib2 using the Intel compilers on Windows is possible but is not supported. (Grib files will be limited to 2 GB, and I have no way to test.)

Note: you To compile with the intel compilers

   export CC=icc
   export FC=ifort
   export COMP_SYS=intel_linux
   make

On HPCs, I wouldn't be surprised that some systems are configured so that you can load a GNU module in order to use to use gcc or you can load an INTEL module in order to use icc/ifort but you can't load both modules at the same time. In this case, the default makefile will not work. The work around is to disable jpeg2000 support or just use gcc/gfortran.

Compiling with Cygwin (Windows)

The only Windows C compiler supported is Cygwin's gcc. The other Windows C compilers follow Microsoft's lead where a "long int" is 32-bits on a 64 bit operating system. Wgrib2 will work with a 32-bit long int but will limit a grib file to an unacceptable 2GB size. Cygwin's gcc also supports POSIX which you don't have to turn off features that require POSIX.

In testing wgrib2 v3.0.0, I found the cygwin was using gnu make v4.3 and the makefile had to be updated. (See step 5 in the linux compile.)

Compiling in MacOS

You need to use compile with gcc and gfortran. The default MacOS installation has gcc pointing to clang. You can get the real deal from homebrew.

With wgrib2 v3.0.0, MacOS support is now builtin. For prior releases, there have been nice pages which detail the compiling process.

Compiling with other compilers

There is no support for other compilers. At one time, AIX was supported until our machines were scraped (2012). Use to support gcc/g95 and the open64 compilers until development was stopped on g95 and open64. The Portland compilers seem to have problems with wgrib2 (2019). I haven't tried the Cray compilers because the Crays are already well supported by gcc/gfortran and icc/ifort.

The default wgrib2 configuration requires both a C and f90+ compiler. In order to combine C and fortran code, the C and Fortran compilers need to be compatible with each other. The supported pairs of compilers are (gcc, gfortran), (icc, ifort) and (clang, flang). The makefile has untested support for (opencc, openf95), (gcc, g95) and and (xlc, xlf).

If there is no supported fortran compiler, wgrib2 can be compiled with only a C compiler. The generated wgrib2 will not support -new_grid and the fortran interface (wgrib2api).

1) download ftp://ftp.cpc.ncep.noaa.gov/wd51we/wgrib2/wgrib2.tgz
2) remove pre-existing grib2 directory if exists: rm -r grib2
3) untar wgrib2.tgz:  tar -xzvf wgrib2.tgz   (use gnu tar)
4) cd to main directory:  cd grib2
5) Update makefile for gnu make version 4.3+ incompatability, needed for wgrib2 upto v3.0.0
    To check the version of make,
       $ make --version
    for version wgrib2 v3.0.0, and gnu make v4.3+, get a new makefile
       $ curl -o makefile "https://ftp.cpc.ncep.noaa.gov/wd51we/wgrib2/makefile_v2_for_wgrib2_v3.0.0"
    for earlier versions of wgrib2 and gnu make v4.3+
       $ sed 's/\\#/#/' makefile.tmp
       $ mv makefile.tmp makefile
6) edit the makefile and change
     USE_IPOLATES=1 -> USE_IPOLATES=0
     MAKE_FTN_API=1 -> MAKE_FTL_API=0
7) define the CC environement variable and make
   Bash:
     export CC=(your C compiler)
     make               (for wgrib2 utility)
     make lib           (for wgrib2 api)
   Csh
     setenv CC (your C compiler)
     make               (for wgrib2 utility)
     make lib           (for wgrib2 api)
8) See if wgrib2 was compiled
     wgrib2/wgrib2 -config

Compiling wgrib2 on OS X. See Grace Peng's blog. Compiling wgrib2 on OS X. See Daniel Bowman's blog

Comments:

  1. For Windows, use Cygwin compilers and bash shell and the linux instructions
  2. For Windows 10, you can use WSL and the Ubuntu compilers. Use the generic linux instructions
  3. Intel and the Portland compilers are not compatible with the Jasper library

Netcdf4

Wgrib2 can be compiled with netcdf4 instead of netcdf3. The advantages of netcdf4 include compression and support for larger files. The cost is that the compiling takes a long time on a weak cpu (hours on an Apollo-Lake cpu), and the wgrib2 executable is about 3 times larger than the netcdf3 version. Consequently the default configuration is to support creation of netcdf3 files. As computers get faster and include more memory, your default configuration should tend towards compiling with netcdf4.
Default makefile:
..
USE_NETCDF3=1
USE_NETCDF4=0
..

Modified makefile for netcdf4
..
USE_NETCDF3=0
USE_NETCDF4=1
..
When you run make, it will fail twice with requests that you download the netcdf4 and hdf5 source codes. After you download the two software packages, you can make wgrib2.

The netCDF4 option is currently unsupported (10/2020). The previous release of hdf5 would not compile with modern gcc compilers. According to the release notes, the current release of hdf5 supports more modern version of gcc but not the latest versions. In personal testing, I had to revert to an older version of hdf5 to make wgrib2 work. It may be a problem with my code. However, the older version will not compile with even a moderately modern gnu compiler. Older versions of ihe Intel compiler were fine, but the latest version had problems (1/2021).

Probably every OS includes precompiled netcdf4 and hdf5 libraries. Modifying the wgrib2 makefile to use the system netcdf4 files will depend on the system and the source of the netcdf4 libraries.

JPEG2000: Jasper, OpenJPEG

With wgrib2 v3.0.0, jpeg2000 compression can be handled by either the Jasper or OpenJPEG library. Both libraries are equally slow. The advantages with the Jasper library is that the last problem with the Jasper library was fixed may years ago. The advantage with the Jasper library is the last problem was fixed many years ago. The disadvantage with Jasper is that support may be lacking. The advantage with OpenJPEG is the support is ongoing. The disadvantage is that it requires cmake to build. Consequently I cannot build and test the OpenJPEG version on my linux workstation at work. So it is suggested that you use the Jasper library unless you have to use OpenJPEG.

Missing zlib.h

I haven't been able to reproduce this compiling problem. The png compression requires both the zlib library and the zlib.h include. I suspect that the libpng configure program is using the system zlib because it finds the system zlib library. Perhaps the fix is to install the zlib.h files by installing the "zlib-develop" package. Another fix is to remove the system zlib and let libpng use the wgrib2-supplied zlib. The third fix is to diable png compression in the make file. PNG compression is not very good and I have only seen it used by RADAR files from ESRL.

wgrib2 compile questions

Question: Warnings about parameter type mismatch in fftpack.f"?

Answer:
   The file fftpack.f is was written to an old fortran standard
before you had an allocatable array and memory was at a premium.
To save space, a real work array was defined and used in some parts 
of the code.  The same array was used in other parts of the code 
which needed an integer work array.  You save memory by using the 
same array in both places.  It was common practice and the old 
compilers didn't complain.  Ignore the warnings.

Question: Why do you not support "make install"?

Answer:

  "Make install" is not supported by the makefile because
installation is just copying the executable to another location.
There is no obvious default for all systems.

Question: Why do you build japser with the following flags?

     --disable-libjpeg --disable-opengl

Answer:

Libjpeg is not needed by wgrib2.  The makefile should work when
the system doesn't have libjpeg installed.

Opengl is not needed by wgrib2. Makefile should still work when the 
system doesn't have opengl installed.

The wgrib2 makefile compiles the libraries in a manner optimized for wgrib2.
The libraries are not meant for general use.  For example, the netcdf library
doesn't have a fortran interface because I want the makefile to
work if a user lacks a fortran compiler. 

Question: Why do build zlib and libpng?

Answer:  Not all linux distributions include libpng or they give it a 
different name.  Some linux distributions may have different versions 
of libpng and zlib.

Question: Why don't you make netcdf4 the default netcdf package?

Answer:

The Netcdf4/hdf5 libraries are very big and the end result is a long
download time and a large executable. If you decide to compile with
Netcdf4, the make will fail with the commands needed to download
the netcdf4 and hdf5 libraries.  After you have download the libraries,
the make should work. 

At the current time (9/2020), compiling with the netcdf4 option has been 
unsupported for a few years.  (Classes of users will have problems and nothing
can be done.) Until recently, the hdf5 library had problems being compiled 
with the newer versions of gcc.  The latest version will compile with the 
gcc included with Ubuntu.  However the newest version breaks the -import_netcdf
option with a uninformative hdf5 error message. 

Question: Why don't you automatically download the netcdf and hdf5 packages?
Answer:

   People concerned about security would not like that action.

Question: I get the error messagee: /bin/sh: ./configure: Permission denied

Answer:

It is possible to configure a filesystem so that the execute bit is
set to zero. Programs and scripts will not execute when they reside
on that filesystem.  Try compiling on another filesystem.  If that doesn't
work, you are probably not allowed to download programs.  The solution
is to ask that your sys admin to compile the program for you.  But if
your sys admin will compile code for you, why are you reading this page?
Let him read this page.

Question: Why the options to turn off aec, jpeg2000 and png compression

Answer:

Some HPC machines use one type of cpu and/or OS to compile and
another type of cpu and/or OS to run (cross-compile).  Using configure
scripts can be a challenge when cross-compiling.  I wanted a reasonable
wgrib2-subset that can be cross-compiled.  Consequently all libraries that
that require a configure script are optional.

The configure programs of various libraries may not handle certain
machines or compilers.  For example an ARM user had to turn off the
jasper library.  Clang on the MacOS may be a problem for the AEC
library.

Question: Why does compiling wgrib2 with the Netcdf option fail?

Answer:

This frequently happens with 32-bit machines.  This problem goes away
on a 64-bit platform or if netcdf-4 is used instead of netcdf-3. This
problem is not being addressed because of a lack of 32-bit machines.

Some versions of hdf5 will not compile with modern C compilers.

Question: Why the option to turn off g2clib?

Answer: g2clib is not needed but it can be installed for testing at NCEP.

Question: Why gctpc and proj4? Why not proj4 without gctpc?

Answer: Gctpc and proj4 are both projection libraries and only one is
really needed.  Proj4 has more functionality, is modern and has an
active support group.  Gctpc is old and is more-or-less unsupported.
On the other hand, gctpc is simpler, works and supports OpenMP like the
rest of wgrib2.  Proj4 supports a different threading model.  As
grids get larger, you want the speed of gctpc. Both libraries are being 
supported in order to help debugging and in the case that a future
grid requires Proj4 support.

Question: Why so many compile options?

Answer:  
  Some systems or compilers do not support certain POSIX features
  Some functionality is not needed by most systems.
  Not all systems have fortran, MySQL, or support NetCDF/HDF5.
  The code that calls IPOLATES is not using a standard interface.
  Some systems do not handle cross-compiling easily.
  Some uses of wgrib2 may require a small RAM usage.
  Same makefile is used to compile the wgrib2 library.
  Some compilers may not support C99 or C11 (future).
 Problem: I compiled wgrib2 and when I run it, it complains about 
      missing libraries.

Answer:
  I have the same problem with our Cray.  If you compile with the Intel 
  compilers, you have to load the Intel environment before running the
  executables.  If you compile with the GNU compilers, you have to load 
  the GNU environment before running.  The way to get around this
  problem is to compile wgrib2 with the same compiler that was used
  to compile the system commands.  (An old version of gcc/gfortran.)
Question: I am compiling with Intel Compilers and amd getting undefined references.

Answer:
  If you get an error message to a "ompc" routine like

      Ncpu.c:(.text+0x24): undefined reference to `ompc_set_num_threads'

  This is because you are using an old Intel compiler.  The makefile uses the 
current option to enable OpenMP rather than the old option.  

NOAA/ National Weather Service
National Centers for Environmental Prediction
Climate Prediction Center
5830 University Research Court
College Park, Maryland 20740
Climate Prediction Center Web Team
Page modified: Dec 29, 2017, Oct 13, 2019, Sep 30, 2020, Oct 2020, Feb 2021
Disclaimer Privacy Policy