Skip Navigation Links www.nws.noaa.gov 
NOAA logo - Click to go to the NOAA home page National Weather Service   NWS logo - Click to go to the NWS home page
Climate Prediction Center
 
 

 
About Us
   Our Mission
   Who We Are

Contact Us
   CPC Information
   CPC Web Team

 
HOME > Monitoring_and_Data > Oceanic and Atmospheric Data > Reanalysis: Atmospheric Data > wgrib2-ncep_norm
 

wgrib2: -ncep_norm

Introduction

The GFS model, for example, produces precipitation with a hard-to-use time step. If you gathered the precip forecasts from the GFS, it would look like,

$ wgrib2 apcp.grb
1:0:d=2009042400:APCP:surface:0-3 hour acc fcst:
2:3340:d=2009042400:APCP:surface:0-6 hour acc fcst:
3:5534:d=2009042400:APCP:surface:6-9 hour acc fcst:
4:7239:d=2009042400:APCP:surface:6-12 hour acc fcst:
5:9304:d=2009042400:APCP:surface:12-15 hour acc fcst:
6:12378:d=2009042400:APCP:surface:12-18 hour acc fcst:
If you obtained the 6 hourly forecasts, everthing would be quite useful. But if you obtained the 3 hourly forecasts, you would notice the averaging period is either 3 or 6 hours. To create 3-hour accumulations, you can use the -ncep_norm option.
$ wgrib2 apcp.grb -ncep_norm new_apcp.grb
1:0:d=2009042400:APCP:surface:0-3 hour acc fcst:
2:3340:d=2009042400:APCP:surface:0-6 hour acc fcst:
3:5534:d=2009042400:APCP:surface:6-9 hour acc fcst:
4:7239:d=2009042400:APCP:surface:6-12 hour acc fcst:
5:9304:d=2009042400:APCP:surface:12-15 hour acc fcst:
6:12378:d=2009042400:APCP:surface:12-18 hour acc fcst:
7:14396:d=2009042400:APCP:surface:18-21 hour acc fcst:
$
$ wgrib2 new_apcp.grb
1:0:d=2009042400:APCP:surface:0-3 hour acc fcst:
2:7769:d=2009042400:APCP:surface:3-6 hour acc fcst:
3:13271:d=2009042400:APCP:surface:6-9 hour acc fcst:
4:18773:d=2009042400:APCP:surface:9-12 hour acc fcst:
5:24275:d=2009042400:APCP:surface:12-15 hour acc fcst:
6:32044:d=2009042400:APCP:surface:15-18 hour acc fcst:
7:38301:d=2009042400:APCP:surface:18-21 hour acc fcst:

The -ncep_norm option is not limited to 3 hour intervals. It can be used with the CFSv2 which has 1 hour intervals. Some restrictions are:

  1. time units of forecast time and statistical processing must be the same
  2. only works with averages and accumulations (Code Table 4.10)
  3. the grid must be the same
  4. the meta-data must be the same except for the forecast timing.
  5. Does not work for CRAIN, CSNOW, CICEP, CFRZR

Usage

-ncep_norm output_file

Conditions

The input grib file must have a very specific format. Each field must be grouped together in time series format. That is because the program scans the file in sequential order. Of course, you can alter the order by the -i option. For example,

$ wgrib2 -match ':APCP:' mixed_up -vt | sort -t: -k3,3 | \
   wgrib2 -i mixed_up -ncep_norm new_apcp.grb
1:0:d=2009042400:APCP:surface:0-0 day acc fcst:
6:26940:d=2009042400:APCP:surface:0-3 hour acc fcst:
11:56805:d=2009042400:APCP:surface:0-6 hour acc fcst:
16:85235:d=2009042400:APCP:surface:6-9 hour acc fcst:
51:281962:d=2009042400:APCP:surface:6-12 hour acc fcst:
61:338477:d=2009042400:APCP:surface:12-15 hour acc fcst:
66:367933:d=2009042400:APCP:surface:12-18 hour acc fcst:
71:396448:d=2009042400:APCP:surface:18-21 hour acc fcst:
...
$ wgrib2 new_apcp.grb
1:0:d=2009042400:APCP:surface:0-3 hour acc fcst:
2:7769:d=2009042400:APCP:surface:3-6 hour acc fcst:
3:13271:d=2009042400:APCP:surface:6-9 hour acc fcst:
4:18773:d=2009042400:APCP:surface:9-12 hour acc fcst:
5:24275:d=2009042400:APCP:surface:12-15 hour acc fcst:
6:32044:d=2009042400:APCP:surface:15-18 hour acc fcst:
7:38301:d=2009042400:APCP:surface:18-21 hour acc fcst:
...

Using forecast files

The typical GFS forecast file has a name like,

GFS:
    gfs.t(HH)z.(type).(resolution).f(fhour)

    gfs.t06z.pgrb2b.0p50.f006
    gfs.t18z.pgrb2.0p50.f006
So to get the "normalized" TCDC, you can do
   $ test.sh  gfs.t18z.pgrb2.0p50.f???

where test.sh is

   #!/bin/sh
   #
   # usage $0 output (list of gfs grib2 forecast files in order)
   #
   A=":TCDC:entire atmosphere:"
   B=":TCDC:low cloud layer:"
   C=":TCDC:middle cloud layer:"
   D=":TCDC:high cloud layer:"
   output=$1
   shift 1
   cat $* | wgrib2 - -match "($A|$B|$C|$D)" -set_grib_type c3 \
      -if "$A" -ncep_norm $output \
      -if "$B" -ncep_norm $output \
      -if "$C" -ncep_norm $output \
      -if "$D" -ncep_norm $output

The above script uses several techniques to speed up the processing.

"cat $*"     This writes the grib data in chronological sequence to stdout.
             (GFS files are in chronological sequence when sorted.)
             This is quite fast because it reads the files sequentially
             and avoids a temporary disk file.

"wgrib2 -"   This reads the grib file from stdin

"-match "($A|$B|$C|$D)"    This avoids decoding grib messages that
             do not match the specification

"-if "$A" -ncep_norm $output "   By using an "-if", the -ncep_norm operator
             sees the data in the correct order.  "-ncep_norm" is an
             output operator, so it terminates the "-if".

The new GFS forecast files (precip, runoff)

The yet-to-be-released GFS files have several quantities that are averaged or accumulated from the start of the forecast rather than strange system that the current GFS is using.

Sample from a masterfile (2/20/2018)
527:73733057:d=2018022000:CPRAT:surface:0-10 day ave fcst:
528:73916984:d=2018022000:PRATE:surface:0-10 day ave fcst:
529:74072009:d=2018022000:APCP:surface:0-10 day acc fcst:
530:74335029:d=2018022000:ACPCP:surface:0-10 day acc fcst:
531:74530375:d=2018022000:WATR:surface:0-10 day acc fcst:
536:74621710:d=2018022000:CSNOW:surface:0-10 day ave fcst:
537:74632230:d=2018022000:CICEP:surface:0-10 day ave fcst:
538:74632532:d=2018022000:CFRZR:surface:0-10 day ave fcst:
539:74633681:d=2018022000:CRAIN:surface:0-10 day ave fcst:

CSNOW, CICEP, CFRZR and CRAIN cannot be made into smaller time intervals. However, we can makc CPRAT, PRATE, APCP, ACPCP and WATR (runoff) into smaller intervals using a modified test.sh (previous example).

   #!/bin/sh
   #
   # usage $0 output (list of gfs grib2 forecast files in order)
   #
   A=":CPRAT:surface:"
   B=":PRATE:surface:"
   C=":APCP:surface:"
   D=":ACPCP:surface:"
   E=":WATR:surface:"
   F=":NCPCP:surface:"
   output=$1
   shift 1
   cat $* | wgrib2 - -match "($A|$B|$C|$D|$E|$F)" -set_grib_type c3 \
      -if "$A" -ncep_norm $output \
      -if "$B" -ncep_norm $output \
      -if "$C" -ncep_norm $output \
      -if "$D" -ncep_norm $output \
      -if "$E" -ncep_norm $output \
      -if "$F" -ncep_norm $output
$ fix_precip.sh $ptmp/junk gfs.t00z.master.grb2f036 gfs.t00z.master.grb2f038 gfs.t00z.master.grb2f039
750:1364093685:d=2018022000:CPRAT:surface:0-36 hour ave fcst:
751:1366549463:d=2018022000:PRATE:surface:0-36 hour ave fcst:
752:1368408145:d=2018022000:APCP:surface:0-36 hour acc fcst:
753:1370627152:d=2018022000:ACPCP:surface:0-36 hour acc fcst:
755:1373685446:d=2018022000:WATR:surface:0-36 hour acc fcst:
1726:3358945351:d=2018022000:CPRAT:surface:0-38 hour ave fcst:
1727:3361410173:d=2018022000:PRATE:surface:0-38 hour ave fcst:
1728:3363276117:d=2018022000:APCP:surface:0-38 hour acc fcst:
1729:3365559288:d=2018022000:ACPCP:surface:0-38 hour acc fcst:
1731:3368697828:d=2018022000:WATR:surface:0-38 hour acc fcst:
2702:5335427590:d=2018022000:CPRAT:surface:0-39 hour ave fcst:
2703:5337896546:d=2018022000:PRATE:surface:0-39 hour ave fcst:
2704:5339766142:d=2018022000:APCP:surface:0-39 hour acc fcst:
2705:5342064567:d=2018022000:ACPCP:surface:0-39 hour acc fcst:
2707:5345230153:d=2018022000:WATR:surface:0-39 hour acc fcst:

$ wgrib2 $ptmp/junk
1:0:d=2018022000:CPRAT:surface:0-36 hour ave fcst:
2:2281003:d=2018022000:PRATE:surface:0-36 hour ave fcst:
3:3998443:d=2018022000:APCP:surface:0-36 hour acc fcst:
4:6059444:d=2018022000:ACPCP:surface:0-36 hour acc fcst:
5:7633622:d=2018022000:WATR:surface:0-36 hour acc fcst:
6:8265949:d=2018022000:CPRAT:surface:36-38 hour ave fcst:
7:11405351:d=2018022000:PRATE:surface:36-38 hour ave fcst:
8:14454386:d=2018022000:APCP:surface:36-38 hour acc fcst:
9:15717551:d=2018022000:ACPCP:surface:36-38 hour acc fcst:
10:16651679:d=2018022000:WATR:surface:36-38 hour acc fcst:
11:17182895:d=2018022000:CPRAT:surface:38-39 hour ave fcst:
12:20530120:d=2018022000:PRATE:surface:38-39 hour ave fcst:
13:23751467:d=2018022000:APCP:surface:38-39 hour acc fcst:
14:24880145:d=2018022000:ACPCP:surface:38-39 hour acc fcst:
15:25692065:d=2018022000:WATR:surface:38-39 hour acc fcst:

See also:


NOAA/ National Weather Service
National Centers for Environmental Prediction
Climate Prediction Center
5830 University Research Court
College Park, Maryland 20740
Climate Prediction Center Web Team
Page last modified: Feb 20, 2018.
Disclaimer Privacy Policy