GrADS allows you use a single data descriptor file to aggregate multiple data files and handle them as if they were one individual file. The individual data files must be identical in the X, Y, and Z dimensions and have the same list of variables. The time range of each individual file must be indicated it its filename. Beginning with version 2.0, data files may also be aggregated in the ensemble dimension.
First, the DSET entry has a substitution template instead of
a filename. See below for a description of all the possible components of the
template. Second, the OPTIONS entry contains the template
keyword. Third, the TDEF entry describes the time range for the
entire set of data files.
Templating works on any GrADS data type for which you can write a descriptor file. If you specify any additional OPTIONS keywords in the data descriptor file, make sure the options apply equally to each file included in the template.
You can use the set misswarn
command to alert you if any of the data files in the templated set is missing.
Valid components of the substitution template for the TIME axis are as follows:
%x1
1 digit decade
%x3
3 digit decade
%y2
2 digit year
%y4
4 digit year
%m1
1 or 2 digit month
%m2
2 digit month (leading zero if needed)
%mc
3 character month abbreviation
%d1
1 or 2 digit day
%d2
2 digit day (leading zero if needed)
%h1
1 or 2 digit hour
%h2
2 digit hour
%h3
3 digit hour (e.g., 120 or 012)
%n2
2 digit minute; leading zero if needed
%f2
2 digit forecast hour; leading zero if needed; more digits added for hours >99; hour values increase indefinitely
%f3
3 digit forecast hour; leading zeros if needed; more digits added for hours >999; hour values increase indefinitely
%fn2
2 digit forecast minute; leading zero if needed; more digits added for minutes > 99; minute values increase indefinitely (2.0.a9+)
%fhn
forecast time expressed in hours and minutes (hhnn
) where minute value (nn)
is always <=59
and hour value (hh
) increases indefinitely. If hh
or nn
are <=9, they are padded with a 0
so they are always at least 2 digits; more digits added for hours >99. (2.0.a9+)
%fdhn
forecast time expressed in days, hours, and minutes (ddhhnn
) where minute value (nn)
is always <=59,
hour value (hh
) is always <=23 and day value (dd
) increases indefinitely. If dd
, hh,
or nn
are <=9,
they are padded with a 0 so they are always at least 2 digits; more digits added for days >99. (2.0.a9+)
%j3
3 digit julian day (day of year) (2.0.a7+)
%t1
1 or 2 digit time index (file names contain number sequences that begin with 1 or 01) (2.0.a7+)
%t2
2 digit time index (file names contain number sequences that begin with 01) (2.0.a7+)
%t3
3 digit time index (file names contain number sequences that begin with 001) (2.0.a7+)
%t4
4 digit time index (file names contain number sequences that begin with 0001) (2.0.a8+)
%t5
5 digit time index (file names contain number sequences that begin with 00001) (2.0.a8+)
%t6
6 digit time index (file names contain number sequences that begin with 000001) (2.0.a8+)
%tm1
1 or 2 digit time index (file names contain number sequences that begin with 0 or 00) (2.0.a7+)
%tm2
2 digit time index (file names contain number sequences that begin with 00) (2.0.a7+)
%tm3
3 digit time index (file names contain number sequences that begin with 000) (2.0.a7+)
%tm4
4 digit time index (file names contain number sequences that begin with 0000) (2.0.a8+)
%tm5
5 digit time index (file names contain number sequences that begin with 00000) (2.0.a8+)
%tm6
6 digit time index (file names contain number sequences that begin with 000000) (2.0.a8+)
When specifying the initial time (e.g., NWP model output), use these substitutions:
%ix1
initial 1 digit decade %ix3
initial 3 digit decade %iy2
initial 2 digit year %iy4
initial 4 digit year %im1
initial 1 or 2 digit month %im2
initial 2 digit month (leading zero if
needed) %imc
initial 3 character month abbreviation
%id1
initial 1 or 2 digit day (leading zero
if needed) %id2
initial 2 digit day %ih1
initial 1 or 2 digit hour %ih2
initial 2 digit hour %ih3
initial 3 digit hour %in2
initial 2 digit minute (leading zero if needed)With the introduction of the extra grid dimension for ensembles in version 2.0, support was also added for file templating over E. The sole substitution template is %e
and the substitution string is the ensemble name, which is provided in the EDEF entry in the descriptor file. Note that the ensemble names are limited to 15 characters -- keep this limit in mind when designing your data directory structure and file naming conventions (or use symbolic links to create short aliases for longer filenames). If you are templating over the ensemble dimension, there can be only one ensemble member per file.
If your data set has an ensemble dimension, and you are using templating over T but not E (i.e., there is no %e in the DSET entry), then all ensemble members are presumed to have identical time axes, and all members must be contained in the data file for a given time. Templating over T but not E is not supported for data sets in flat binary or GRIB1 formats.
The %ch
template option, introduced in version 1.9b4, allows
for any user-specified string substitution, not just date strings.
This is useful when none of the above template options match the time ranges in
the files you wish to aggregate, or if the files are located on different disk pathnames.
The syntax is as follows:
%ch
substitute string
%ch
template in your DSET entry, then you also need
to put additional CHSUB entries
in the descriptor file that contain two integers (t1 and t2) followed by a string
which will be substituted for %ch
in the data file names for the
time steps beginning withCHSUB t1 t2 string
Version 2.1.a3 adds a new feature to the string substitution template: the string
provided in the CHSUB
entry may contain time-based template components. GrADS will do the CHSUB
string substitution before the complete filename is generated by resolving all the other template substitution components. An application of this strategy might be to merge a reanalyis and forecast into one seamless time series, but the reanalyes and forecasts have different file naming conventions. See example #6 below.
1may92.dat
2may92.dat
...
31may92.dat
DSET ^%d1may92.dat
OPTIONS template
TDEF 744 linear 0z1may1992 1hr
1jun92.dat
2jun92.dat
...
1jan93.dat
DSET ^%d1%mc%y2.dat
OPTIONS template
TDEF 6624 linear 0z1may1992 1hr
pr.1880_1889.nc
pr.1890_1899.nc
pr.1900_1909.nc
pr.1910_1919.nc
pr.1920_1929.nc
pr.1930_1939.nc
pr.1940_1949.nc
DSET ^pr.%x30_%x39.nc
OPTIONS template
DTYPE netcdf
TDEF 840 linear jan1880 1mo
pr.1851-1900.nc
pr.1901-2000.nc
DSET ^pr.%ch.nc
CHSUB 1 600 1851-1900
CHSUB 601 1800 1901-2000
OPTIONS template
DTYPE netcdf
TDEF 1800 linear jan1851 1mo
DSET %ch
CHSUB 1 600 /disk1/pr.1851-1900.nc
CHSUB 601 1800 /disk2/pr.1901-2000.nc
MMOUT_DOMAIN1_00
MMOUT_DOMAIN1_01
MMOUT_DOMAIN1_02
so your DSET enry will look like this: DSET ^MMOUT_DOMAIN1_%tm2
DSET /data/projects/ops/pub/%ch
CHSUB 1 1898 das/Y%y4/M%m2/D%d2/GEOS.fp.asm.inst3_2d_smp_Nx.%y4%m2%d2_%h2%n2.V01.nc4
CHSUB 1899 1909 forecast/Y2014/M10/D15/H06/GEOS.fp.fcst.inst3_2d_smp_Nx.20141015_06+%y4%m2%d2_%h2%n2.V01.nc4
OPTIONS template