Climate Prediction Center - wgrib2: -ndates

www.nws.noaa.gov

About Us



Contact Us

HOME > Monitoring_and_Data > Oceanic and Atmospheric Data > Reanalysis: Atmospheric Data > wgrib2-ndates

wgrib2: -ndates

Introduction

The -ndates option creates a list of date codes using a do-loop syntax. This option has nothing to do with grib, but adding it to wgrib2 was trivial, and this option has been so helpful in scripting. Anyways, if a Swiss army knife can have a bottle opener, wgrib2 can have a date code routine. Anyways one less program to port is helpful.

Note that until wgrib2 v3.0.3, the number of date codes was limited by the stdout buffer size. (See wgrib2 -config.) Wgrib2 v3.1.4 allows the optional equal sign option to include the terminating date code.

The -ndate and -ndates options are initialization routines. They are run when all the options are initialized. Therefore they are run before reading the grib file. The -ndates_fmt option has to preceed the -ndates option to alter the output format.

The wgrib2 command line can have multple -ndates options.

-ndates DATE DATE2 DT2     lt form
-ndates DATE DT1 DT2       lt form
-ndates DATE =DATE2 DT2    le form     wgrib2 v3.1.4
-ndates DATE =DT1 DT2      le form     wgrib2 v3.1.4

 DATE, DATE2 = YYYY, YYYYMM, YYYYMMDD, YYYYMMDDHH, YYYYMMDDHHmm, YYYYMMDDHHmmss
 DT1, DT2 = (integer)(unit)  unit=yr,mo,dy,hr,mn   (same as GrADS)
            note: DT must be positive and greater than zero

If DT1 is given, then DATE2=DATE+DT1

lt form:
    C-meta:          for (date=DATE; date < DATE2; date += DT2) print date;
    Fortran-meta:    date=DATE
  		     while (date < DATE2)
                        print date
                        date=date+DT2
                     endwhile

le form:
    C-meta:          for (date=DATE; date<= DATE2; date += DT2) print date;
    Fortran-meta:    date=DATE
                     while (date <= DATE2)
                        print date
                        date=date+DT2
                     endwhile

The only differences between the lt and le forms is the do-loop test clause.


DATE=integer of the form YYYY, YYYYMM, YYYYMMDD, YYYYMMDDHH, YYYYMMDDHHmm, YYYYMMDDHHmmss
                note: leading zeros cannot be replaced by spaces
                YYYY = year (0000..9999), there were changes in the calendar (ex in 1752) which are ignored
                MM = month (01..12)
                DD = day (01..31)
                HH = hour (00..23)
                mm = minute (00..59)
                ss = second (00..59), there are no provisions for leap seconds
DT = (integer)(time unit)           "time unit" follows the GrADS convention
               integer > 0
               time unit = yr (year)
                           mo (month)
                           dy (day)
                           hr (hour)
                           mn (minute)
                           there is no seconds time unit defined by GrADS

One useful feature of -ndates is the format of the output date is appropriate for the input date code and DT2. The format has the most precision for both the input date code and DT2.

      Priority of the output format of the date codes (high to low)
   output date code: YYYYMMDDHHmmss      if input date code is YYYYMMDDHHmmss
   output date code: YYYYMMDDHHmm        if input date code is YYYYMMDDHHmm   or DT2 is in minutes (mn)
   output date code: YYYYMMDDHH          if input date code is YYYYMMDDHH     or DT2 is in hours (hr)
   output date code: YYYYMMDD            if input date code is YYYYMMDD       or DT2 is in days (dy)
   output date code: YYYYMM              if input date code is YYYYMM         or DT2 is in months (mo)
   output date code: YYYY                if input date code is YYYY           or DT2 is in years (yr)

Examples

Suppose you want a list of days for 2003.

$ wgrib2 /dev/null -ndates 2003 2004 1dy
 20030101 20030102 ... 20231231

 You could get the same result by
$ wgrib2 /dev/null -ndates 2003 1yr 1dy

Now suppose you want the output to include the hours, then

$ wgrib2 /dev/null -ndates 2003 2004 24hr
 2003010100 2003010200 ... 2023123100

Suppose that you want the list of day from April 2000 to December 2000 inclusive. Then you can use the inclusive form of -ndates.

$ wgrib2 /dev/null -ndates 200004 =200012 1mo
 200004 ... 200012

Usage

  lt form
-ndates DATE DATE2 DT2
-ndates DATE DT1 DT2
  le form
-ndates DATE =DATE2 DT2
-ndates DATE =DT1 DT2

Usually wgrib2 is given the null file of "/dev/null".  For windows,
try using NUL.

    DATE: YYYYMMDDHH or YYYYMMDDHHmm or YYYYMMDDHHmmss
    DATE2: if DT1 is given, then DATE2 = DATE + DT1

    DT1: ending dt
    DT2: step dt
    dt: (integer)Time_unit,   integer > zero
    Time_unit: yr (year)
               mo (month)
               dy (day)
               hr (hour)
               mn (minute)

    prints a list of date codes

    C-meta:    for (date=DATE; date < DATE2; date += DT2) print date;
    C-meta:    for (date=DATE; date <= DATE2; date += DT2) print date;

    the output date code has the precision necessary for DATE and DT2 and 
    is converted into a string.  The date is printed out using the 
    "ndates_fmt" format.  The default format is " %s".  (C format)

Example

        print date codes for 1 month, every day

$ wgrib2 /dev/null -ndates 2019020100 1mo 1dy
 2019020100 2019020200 2019020300 2019020400 2019020500 2019020600 2019020700 2019020800 
2019020900 2019021000 2019021100 2019021200 2019021300 2019021400 2019021500 2019021600 
2019021700 2019021800 2019021900 2019022000 2019022100 2019022200 2019022300 2019022400 
2019022500 2019022600 2019022700 2019022800

$ wgrib2 /dev/null -ndates 201902 1mo 1dy
 20190201 20190202 20190203 20190204 20190205 20190206 20190207 20190208 20190209 20190210 
20190211 20190212 20190213 20190214 20190215 20190216 20190217 20190218 20190219 20190220 
20190221 20190222 20190223 20190224 20190225 20190226 20190227 20190228

        print date codes for 1 day, every 6 hours

$ wgrib2 /dev/null -ndates 2019020100 1dy 6hr
 2019020100 2019020106 2019020112 2019020118

$ wgrib2 /dev/null -ndates 20190201 1dy 6hr
 2019020100 2019020106 2019020112 2019020118

$ wgrib2 /dev/null -ndates 20190201 1dy 6hr
2019020100 2019020106 2019020112 2019020118

$ wgrib2 /dev/null -ndates 2019020100 1dy 6hr
 201902010000 201902010600 201902011200 201902011800

      print the months from 202001 to 202012

$ wgrib2 /dev/null -ndates 202001 202101 1mo
 202001 202002 202003 202004 202005 202006 202007 202008 202009 202010 202011 202012
$ wgrib2 /dev/null -ndates 202001 12mo 1mo
 202001 202002 202003 202004 202005 202006 202007 202008 202009 202010 202011 202012

Finding the Julian date (ordinal date)

The ordinal date is the year and day of the year ranging from 1 and 366. To get the day of the year from YYYYMMDD, you can use -ndates.

wgrib2 /dev/null -ndates {YYYY-1}1231 {YYYYMMDD} 1dy | wc -w
wgrib2 /dev/null -ndates {YYYY} ={YYYYMMDD} 1dy | wc -w               wgrib2 v3.1.4

Date from the Julian date (ordinal date)

Using wgrib2 to convert the date code to the Julian day is easy.

year YYYY
julian_date N

wgrib2 /dev/null -ndate {YYYY} {N-1}dy`

Changing the ndates Format (for xargs)

The output of -ndates has the format,

( DATE)*
space followed by the date, repeated N times.
 for example
 2001010100 2001010106 2001010112 ...

For must uses, this format is fine. However, the xargs program wants each date to be on its own line.

(DATE1)
(DATE2)
(DATE3
...

To convert the ndates format, use sed.

$ dates=`wgrib2 /dev/null -ndates 200001 1mo 1dy | sed 's/^ //' | sed 's/ /\n/g'`
    The first sed removes the leading space.
    The second sed changes all the blanks to newlines
       Note \n is the backslash character followed by the character n.
       This is the bash convention for single quoted strings.  
$ echo "$dates"
    Should see the dates on its own line.
$ echo "$dates" | xargs -P4 -t -I% cp $dir/pgb_%_ensmean $out_dir/
    xargs -P4  run using 4 processes
          -t   echo the command to be executed
	  -I%  The % character is replaced by the line from stdin

You could also change the format by the -ndates_fmt option

$ dates=`wgrib2 /dev/null -ndates_fmt "%s\n" -ndates 200001 1mo 1dy`
$ echo "$dates" | xargs -P4 -t -I% cp $dir/pgb_%_ensmean $out_dir/

Length Limitations of wgrib2 v2.8.0-v3.0.2

In old versions of wgrib2 (v3.0.2 and older), -ndates option was limited in the number of dates it could write before it overflowed the stdout buffer (30KB in v2.0.8 and 100KB in v3.0.0). That should be enough for any reasonable person. However, I was processing years of hourly data and overflowed the buffer. So wgrib2 v3.0.3 writes directly to stdout and avoids the overflow.