IPWG Algorithm Documentation for 3B40RTjjanowiakjjanowiak212007-05-02T14:53:00Z2007-05-02T14:53:00Z11515863771201013211.5604 IPWG Algorithm Documentation for 3B40RT George J. Huffman 27 April 2007<NAME>TRMM HQ (3B40RT)<ALGORITHM DESCRIPTION>This algorithm provides a merger of all available SSM/I and TMI microwave precipitation estimates into a "high-quality" (HQ) precipitation estimate. The SSM/I estimates are computed with the GPROF 6.0-SSMI algorithm in NASA/GSFC Code 613.1 and the TMI estimates are computed with the GPROF 5.0-TMI algorithm (the real-time TRMM 2A12 product). Before merger the SSM/I are calibrated to the TMI using separate global land and ocean histograms based on coincident time-space matched data.Digital data: ftp://trmmopen.nasa.gov/pub/merged/combinedMicroftp://ftp-tsdis.nasa.gov/mergedExample GIF images and QuickTime movies:http://trmm.gsfc.nasa.govInteractive Web-based display and analysis system:http://lake.nascom.nasa.gov/tovas/Detailed documentation (3B4XRT_doc) and programming examples:ftp://trmmopen.nasa.gov/pub/merged/softwareReference:Huffman, G.J., R.F. Adler, D.T. Bolvin, G. Gu, E.J. Nelkin, K.P. Bowman, Y. Hong, E.F. Stocker, D.B. Wolff, 2007: The TRMM Multi-satellite Precipitation Analysis: Quasi-Global, Multi-Year, Combined-Sensor Precipitation Estimates at Fine Scale. J. Hydrometeor., 8(1), 38-55.<SPECTRAL INTERVALS AND APPLICABLE SATELLITES>The input to HQ (3B40RT) consists of pre-computed precipitation estimates based on single-satellite passive-microwave data. At present we employ the GPROF to create the input estimates, and in common with other physically-based algorithms, GPROF uses all available bands from modern conic-scan passive microwave imagers:* 7 channels on DMSP F-13,F-14,F-15 Special Sensor Microwave/Imager (SSM/I)* 9 channels on TRMM Microwave Imager (TMI)The approach is equally applicable to other "high-quality" precipitation estimates, e.g. from Advanced Microwave Scanning Radiometer (AMSR), TRMM Precipitation Radar or Advanced Microwave Sounding Unit-B (AMSU-B) data. For each such new set of input precipitation estimates, the key step is to develop a calibration between the reference set of precipitation estimates (currently TMI-based) and the new set. Taking TMI precipitation as the standard for the sake of illustration, preliminary work indicates that the histograms of PR and AMSU-B rain rate estimates at the 0.25x0.25-deg scale require more aggressive calibrations than are currently applied to SSM/I estimates.<SPATIAL SCALE>0.25x0.25-deg latitude/longitude<TEMPORAL SCALE>3 hours<ANCILLARY DATA>Land/ocean surface type data<ADDITIONAL COMMENTS>IntroductionThe HQ is the first stage of a system to produce the "TRMM and OtherData" estimates in real time. The system was developed to apply new concepts in merging quasi-global precipitation estimates and to take advantage of the increasing availability of input data sets in near real time. The overall system is referred to as the real-time TRMM Multi-Satellite Precipitation Analysis (TMPA-RT). The TMPA-RT is run quasi-operationally on a best-effort basis at the TRMM Science Data and Information System (TSDIS), with on-going scientific development by the research team led by Dr. Robert Adler in the GSFC Laboratory for Atmospheres. Estimates are posted to the web about 6 hours after observation time, although processing issues may delay or prevent this schedule. Due to the experimental nature of these estimates, users are encouraged to report their experiences with the data, and they should expect episodic upgrades or outages as the system develops.File ContentsEach file starts with a header that is one 2-byte-integer row inlength, or 2880 bytes. The header is ASCII in a "PARAMETER=VALUE"format that makes the file self-documenting (e.g., "algorithm_id=3B40RT").Thereafter five data fields follow. All the fields are on a 0.25-deglat./long. grid that increments most rapidly to the east (from thePrime Meridian) and then to the south (from the northern edge). Gridbox edges are on multiples of 0.25 deg. The data fields are writtenas binary data in big-endian byte order. The data fields are: precipitation (2-byte integer) precipitation_error (2-byte integer) total_pixels (1-byte integer) ambiguous_pixels (1-byte integer; highly uncertain values) rain_pixels (1-byte integer)All fields are 1440x720 gridboxes (0-360 deg. E, 90 deg. N-S). Thefirst grid box center is at (0.125 deg. E, 89.875 deg. N). Files areproduced every 3 hours on synoptic observation hours (00 UTC, 03 UTC,..., 21 UTC) as an accumulation of all HQ swath data observed within90 minutes of the nominal file time. Estimates are only computedfor the band 70 deg. N-S.Note that we use the term "gridbox" to denote the values on Level 3data (i.e., gridded data), while we use the term "pixel" to denoteindividual values of Level 2 data (i.e., instrument footprints).Thus, there can be many pixels contributing to a gridbox.Both precipitation and random error are scaled by 100 beforeconversion to 2-byte integer. Thus, units are 0.01 mm/h. To recoverthe original floating-point values in mm/h, divide by 100. Missingsare given the 2-byte-integer missing value, -31999. The remainingfields are in numbers of pixels.Currently the random error fields are all set to the 2-byte-integermissing value, -31999. This placeholder will be replaced with actualestimates as development proceeds.The variable ambiguous_pixels is the count of pixels for which thealgorithm cannot determine whether the scene has valid or invaliddata. It is a subset of the total_pixels and many, but not all, areincluded in raining_pixels. In general, a "high" fraction ofambiguous_pixels indicates that the grid box value is invalid.The originating machine on which the data files are written is aSilicon Graphics, Inc. Unix workstation, which uses the "big-endian"IEEE 754-1985 representation of 4-byte floating-point unformattedbinary numbers. Some CPUs, including PCs and DEC machines, mightrequire a change of representation (i.e., byte swapping) before usingthe data. In some cases, the gunzip routine, used to uncompress thedata, will change representations automatically.Dataset ValidationThese datasets represent a new initiative and should be consideredexperimental. Formal validation studies are underway, but are not yet available. The primary limitations on the HQ (3B40RT) are the sparse sampling by the collection of passive-microwave satellites and algorithm drop-outs in regions with icy or frozen surface. We encourage early users to report successes and problems in applying these datasets to their particular applications.Dataset StatusBeta testing began in early December 2001. An official(experimental) version was instituted in late January 2002. Processing changes occurred on 6 February and 12 March 2002. Theambiguous screening was upgraded for the HQ (3B40RT) as of 09Z28 February 2003 and for the VAR (3B41RT) as of 00Z 2 March 2003.The GPROF estimates for SSM/I over land and coast were upgradedon 12 February 2004. Fractional coverage by precipitation, volumerain, and ambiguous screening upgrades were made to the calibrationof other microwave estimates to the TMI, and cold land and highrainrate improvements were made to the IR calibration on 31March 2004. The Version 6 TMI-GPROF was instituted in the RT starting around 16Z 4 January 2005 (which affects the calibrationfor the entire RT suite). Beginning 07Z 3 February 2005 AMSR-E and AMSU-B estimates were introduced in 3B40RT and the calibrations for 3B41RT were recomputed every 3 hr (but still using an approximate trailing month of match-ups). The GPROF estimates for SSM/I were upgraded to correctly screen bad input values late on 9 March 2005.Users should anticipate a series of versions as the algorithm is developed further. The present areas of interest are: calibrating the RT to be approximately unbiased with respect to the Version 6; improving the HQ product by auditing out AMSU-B data that are deficient in precipitation coverage; and moving to shorter-intervalestimation periods to more accurately represent the time series ofprecipitation.Example ProgramsThe data fields are all written with C-language code as blocks ofbytes, so there are no extraneous bytes in the files. Because thefirst two fields are 2-byte integers and the rest are 1-byte integersin each file (to save space), users must exercise care in usingFORTRAN direct access to read the data. The FORTRAN example programs read all fields with a single OPEN. Alternatively, the files can beopened with different logical record sizes depending on whether one is reading 2-byte-integer or 1-byte-integer fields. Note as well that the units of the logical record size is not part of the FORTRAN 77 standard. On SGI machines it is in 4-byte words, but some other systems expect it in bytes. Also, to repeat an earlier comment, the originating machine on which the data files are written is a Silicon Graphics, Inc. Unix workstation. It uses the "big-endian" IEEE 754-1985 representation of 4-byte floating-point unformatted binarynumbers, and some CPUs, such as PCs and DEC machines, might require a change of representation (i.e., byte swapping) before using the data.The FTP site ftp://trmmopen.nasa.gov/pub/merged/software provides several example programs: read3B4XRT.c C example read_header.f FORTRAN header-read example read_rt_file.f FORTRAN single-read example read_rt_file.pro IDL example read_rt_lines.f FORTRAN line-by-line example<COMPLETE NAME OF CONTACT PERSON>George J. Huffman<E-MAIL ADDRESS OF CONTACT PERSON>huffman@agnes.gsfc.nasa.gov