GrADS 1.7 What's New -- the delta between 1.5.1 and 1.7

Mike Fiorino
fiorino@llnl.gov

Program for Climate Model Diagnosis and Intercomparison
Lawrence Livermore National Laboratory
University of California
P.O. Box 808 L-264
Livermore, CA 94551
510-423-8505 (voice) 510-422-7675(fax)


19 November, 1997

Introduction

I has been over two years since the Mother of All GrADS documents for Version 1.5.1 was put together by myself and Tom Holt of UEA, UK.

A great deal has happenend to GrADS in the mean time and this document will describe new and previous undocumented features of 1.7.

Summary of Changes



Major Changes to gribmap

Highlights of the changes to gribmap:

Automatic Setting of Century for years < 100 in tdef has been disabled

The tdef card in the data descriptor ".ctl" file will set the century to 1900 if the year in the tdef line is >50 and < 100 and 2000 if < 50. Thus,

tdef 1 linear 00z1jan79 12hr

is changed internally to

tdef 1 linear 00z1jan1979 12hr

The COARDS convention requires use of the year "0" for climatology (this is somewhat ridiculous as there is NO year "0" and CDC uses "1" instead). In support of this convention and the GrADS proposed climatology convention, I needed to be able to set years explicitly to values less than 100. Additionally, I think this implicit century setting is a bad idea with the approaching change of the millenium.

However, to maintain backward compatibility with the "old" system, yet allow setting of years < 100, a four character year code must be used in the tdef card, e.g., year 1 = tdef 1 00z1jan0001 1yr and in time expression (e.g., set time 1jan0001. The routine which converts time strings to the GrADS time structure (adtprs in gautil.c is used throughout GrADS (e.g., set time jan1979). Hence, I use a "global" variable in GrADS call mfcmn.fullyear to set the conversion in the old system of 00 -> 2000. This global variable is set whenever a time expression in GrADS or the .ctl is made.

The bottom line is that you should use 4-digit years for ALL year, including 0001.

Support for "INTEGER*2"

While the bit structure of character data (bytes) is well defined and standard, the "INTEGER*2" format is not. Nonetheless, I have received a few requests to support such data types and I have put my best guess at how INTEGER*2 works.

On the positive side, as far as I can tell, this format is machine independent. That is, the INTEGER*2 files can be viewed on any GrADS platform (it does byteswapping although I have not tested this too much). For example, I viewed some NCEP CPC GPI data on the cray as well as big endian workstations.

However, if you write the data using FORTRAN binary mode (sequential on options card in the .ctl file and in the FORTRAN code write(10) a ) then machine independence goes away.

Here's an example .ctl file

dset ^GMS.961025.i2.dat
options yrev
undef -9999.0
xdef 100 linear 90.5 1.0
ydef 100 linear -49.5 1.0
zdef 27 levels 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
tdef 1 linear jan1986 5dy
vars 1
cls 27 -1,40,2,-1 GPI classes 27 is the rainfall
endvars
The key is the -1,40,2,-1 in the units section of the variable description.
-1 special data format for this variable
40 integer data flag
2 2-byte integers data flag
-1 SIGNED INTEGER*2 (-32768 to 32767)
1 UNSIGNED INTEGER*2 (0 to 65535)

New Map data sets and functions

The new map data sets in GrADS 1.6.? are machine independent and support different vector types. This allows different map elements, such as rivers, road, political boundaries, etc., to be differentiated in the map background data set by a type value, which ranges from 0 to 255.

When the map backgrounds are displayed with GrADS, the different map types can be displayed with different colors, thicknesses, and line styles. By default, map types 0, 1, and 2 are enabled for display; 3 to 255 are disabled. Also by default, the map background color, linestyle, and thickness is controlled using the set map command.

The assignment of map types for any particular map background data set is completely up to the data set creator. The map backgrounds provided with GrADS are set up so that map type 0 is coastlines, map type 1 is country boundaries, and map type 2 indicates stat boundaries. The set poli command will enable or disable the display of map types 1 and 2.

As you may have notices, some of the peculiarities of this set up are to maintain upward compatibility with the older map background control commands -- which all still work the same way.

The new command to control map background behavior is the set mpt command:

set mpt type color style thickness

or

set mpt type off

The type is the map type; it can be a number from 0 to 255, or it can be an asterick (*) to indicate this command applies to all the type values. The colar can be set to -1, which indicates to GrADS to use the set map settings for this map type, rather than the settings specified by the set mpt command.

So by default, you get this:

set mpt * off
set mpt 0 -1
set mpt 1 -1
set mpt 2 -1

Lets say you want to use the hires data set and plot political boundaries, but not stat boundaries:

set mpt * off
set mpt 0 -1
set mpt 1 -1

Now, you want to use the hires data set, and have coastlines be thicker, and a different color, than political boundaries:

set mpt * off
set mpt 0 1 1 6
set mpt 1 15 1 1

The set mpdraw and draw map commands work as before; you can do some interesting line types by overlaying:

...
set mpt 51 7 1 12
draw map
set mpt 51 0 1 12
draw map

This would produce two yellow lines parrallel and close together for map type 51.

FORTRAN written station data -- options sequential support in the station map .ctl file

The full document gives more details and examples.

Help at the command line for grads and gxps, gxtran, gribmap, gribscan utilities

Running all the GrADS programs with the -help command line option will now give a listing of all the options (even the ones the average user should probably not use).

Version displayed in window title

The version number set during the build is now displayed in the title of the graphics window. You can also find the version by doing a,

q 
which now displays all the query functions.

undefine -- Freeing Memory taken by defined grids

With the ability to create variables (inside GrADS via the C routine "malloc") using (define) and form collections using (collect), system memory could become a real problem. undefine allows you to return that memory to the system (inside GrADS via the C routine "free"). Simply,

undefine GGGG where GGGG is a defined variable.

For example,

set lon 0 360
set lat -90 90
newgrid=regrid2(u*u-u(t=1),2.5)
d newgrid
undefine newgrid

New behaviour for constant grids

A constant grid displays the string "constant field. Value = VVV" (where VVV is the value) in the graphics display window, but in the case of displaying defined and undefined points, I would have to manually

set gxout fgrid
set fgvals VVV 1 (VVV is the constant value and one is the color 1)
d GGG (GGG is the grid expression)
set gxout contour

Like most things, laziness was the motivator. I now make grads do this automatically, but I also restore the previous display mode (e.g., contour, shaded, etc.).

New and Improved Average functions:
To aave add amean, asum, asumq
To ave add mean, sum, sumq

I have overhauled the code that does the averaging to allow extensions and new behaviours. I want to stress, however, that the original code for the aave and ave functions was not modified; only how the code is called.

Aribtrary Cross Sections -- the collect functions or

"Arbitrary cross sections" is perhaps the most requested new feature of the user community (My own is script libraries). Brian's implementation is very general and allows for more applications than the just arbitrary cross sections.

The basic idea is to transform station data (obs) to gridded data so as to take advantage of the GrADS grid display and analysis features. But we also need to convert gridded data (vertical profiles from a 3-D (lon,lat,lev) model solution) to station data to compare a model sounding(s) to an observations (station data).

The first step was to extend the existing grid-to-station interpolator gr2stn, to pull "station" data from the grids more generally. Now you can, interpolate the gridded data to a station sounding or time series or pull a gridded data sounding or time series from an arbitray (lon,lat), i.e.,

gr2stn(gridexpr,stnexpr)

or

gr2stn(gridexpr,lon,lat)

The interpolation will only work when Z or T are varying (i.e., only one varying dimenion).

We now need to form a "collection" of 1-D data (Z or T varying, grid or station) before we can do any processing/display. The collect command does this job and in is similar to define in that GrADS is putting aside memory for the data.

The second step was to improve the collect function which saves station data profiles or time series (either real station data or gridded data converted to station data using gr2stn) in memory as a set. The command format is:

collect cnum expr

where cnum is the collection number, a value from 0 to 31, and expr is an GrADS expression that gives a station data result (either from station data or grid interpolated). The collect command will only work when one dimension is varying, either Z or T. Each time the collect command is issued for a particular collection number, the station data is added to the collection of stations. The order in which stations are addes IS important. The memory of a collection is returned (freed) by:

collect cnum free

This frees all storage and empties the collection.

The third stat is to convert the collection of station data to grids for display, etc. using the new function coll2gr by:

coll2gr(cnum):

or

coll2gr(cnum,num):

or

coll2gr(cnum,-u):

where the first argument, cnum, is a collection number. The 2nd argument specified the number of vertical levels desired in the output grid; -u indicates that the level should be a union of all levels in the collection of station data. coll2gr will currently only work when the collection of stations is a collection of vertical profiles. Support will be provided where time is the varying dimension. coll2gr produces an output grid that varies in X and Z, so that dimension environment used when coll2gr is invoked should be X and Z varying. The output grid will be scaled such that the station profiles will be represented along the X axis, and will be equally spaced within the range of the dimension environment for the X dimension. The Z dimension of the output grid will range within the range of the Z dimension in the current dimension environment, with the specified number of levels or a union of levels. Station data points outside of the range of levels will be used for interpolating to within the range if appropriate.

Here is a sample script using these features:

*********************************************************************
* The following lines will display an arbitrary X section
* from one specified point to another.  
*
* lon1 is the westernmost longitude point
* lon2 is the easternmost longitude point
* lat1 is the latitude that corresponds to lon1
* lat2 is the latitude that corresponds to lon2
*
* The loop is used to interpolate between points in
* the arbitrary cross section.  This code will plot
* any cross section as long as you specify the points.  
* My code plots cross sections of PV after I calculated
* PV on 11 pressure surfaces.  I have another script
* that plots cross sections of potential temperature, and
* the code is very similar to this, except theta is substituted
* for PV.
*
* Many thanks to Brian Doty at COLA for his help with this code.
*
********************************************************************

'open pv.ctl'
'set grads off'
'set zlog on'
'set x 1'
'set y 1'
'set lev 1000 100'
lon1 = -95.0
lon2 = -90.0
lat1 = 55.0
lat2 = 15.0
lon = lon1
'collect 1 free'
while (lon<=lon2)
  lat = lat1 + (lat2-lat1)*(lon-lon1) / (lon2-lon1)
  'collect 1 gr2stn(pv,'lon','lat')'
  lon = lon + 1
  say lat
  say lon
endwhile

'set x 14 16'
set xaxis 'lon1' 'lon2'
'set clab on'
'set gxout shaded'
'set clevs 0 .5 15'
'set ccols 0 0 7 0'
'd coll2gr(1,-u)'
'set gxout contour' 
'set cint .5'
'd coll2gr(1,-u)'

The X dimension set up for the output grid of coll2gr is artificial in terms fo the world coordinates. Various ways of dealing with this are being worked on, but for the meantime, a way to completely control the labelling of the output is provided:

set xlabs lab1 | lab2 | lab3 | ...
set ylabs lab1 | lab2 | lab3 | ...

Each label string may include blanks. The labels are plotted equally spaced along the indicated axis (spacing can be modified by having blank strings).

For example, suppose we have opened a file with lon,lat data at 10 levels:

Format control over the X and Y labels

The xlabs and ylabs set options allow the x and y axis labeling to be overidden. The syntax is,
set xlabs lab1 | lab2 | lab3 | ...
set ylabs lab1 | lab2 | lab3 | ...

Each label string may include blanks. The labels are plotted equally spaced along the indicated axis (spacing can be modified by having blank strings, e.g., set xlabs | | | | |lab1| |).

Dropdown Menu Widget

A new widget can be used in version 1.6; the dropmenu. This feature is still being worked on, and it is quite likely that the implementation will change in some details before being finalized.

The command to display a dropmenu is similar to the 'draw button' command:

draw dropmenu num x y width height text

where num is the dropmenu widget number (0 to 64); x and y are the center location for the 'base' of the dropmenu; width and height are the size of the 'base' of the dropmenu. The 'base' for the dropmenu looks much like a button, except an italicized font is used. The text parameter specifies the contents of the menu, seperated by vertical bars (|). An example is:

draw dropmenu 1 1 8.3 2 0.4 Variable |Winds (m/s) |Temperature| SLP| Dewpoint

The first item in the text list is the string put in the 'base' of the dropmenu (the base being the part that always appears); the rest of the text are the menu items. Note that blanks are allowed in the strings.

When the user clicks on the 'base', the rest of the menu appears. The colors are somewhat controlled by the 'set button' command; there will be a 'set dropmenu' command implemented soon to allow more control over the colors.

There is currently no way to 'cascade' the menus. Support for this is planned. It may require the syntax of the command to change.

When the script issues the 'q pos' to wait for an event, and a dropmenu is clicked on, the text response is:

Position = x y mb 3 dmn inum

where:
x, y mouse position at the first click
mm mouse button number (1, 2, or 3 for left, middle right)
3 widget class number for dropmenus
dmn drop menu number selected
inum menu item number selected If 0, no item was selected.

Implied define and run

A new convience feature is implied define and run, e.g.,

define a=sqrt(u*u+v*v)

can be done using

a=sqrt(u*u+v*v)

Similarly,

run cbarn.gs

can be done by

cbarn


(Note the standard ".gs" extension for GrADS scripts was dropped too)

Initial Dimension Environment

setting of the initial environment

Plotting Feature

'set arrowhead XXX'

XXX = size in inches if XXX id < 0 then XXX is the max size and
the arrowhead is scaled as the length

'set arrowhead -0.15'

looks nice to me

"global" keyword in dimensions for aave, amean and scorr

I got tired of typing,

aave(grid,lon=0,lon=360,lat=-90,lat=90)

to do a global area average.

Now you can just,

aave(grid,g) or aave(grid,global)

to do the same thing where grid is a grid expersion.

This also works for the function amean (area average WITHOUT weights) and scorr (spatial correlation)

The outxwd command -- direct output of the graphics window to a file

To output the graphicw window to a file in the X windows dump format (XWD) use this command

outxwd filename

where filename is the name of the XWD file

This file will probably not be too useful. I use Image Magick for conversion choirs and to convert to GIF use the convert utility,

convert xwd:filename gif:filename.gif

does the trick for me.

LATS Interface to output GRIB and/or netCDF grids from within GrADS

PCMDI has developed a software library for creating gridded data in either the WMO GRIB format and/or the UNIDATA netCDF format. This library, called the "Library for AMIP II data Transmission Standards or LATS, was designed for outputting global model data, but is sufficiently general so that it supports a wider variety of grids and data types.

Most significant to the GrADS community, LATS creates GRIB and netCDF DATA which is directly readible by GrADS. In the case of GRIB, LATS creates the GrADS ".ctl" data descriptor file and even does the gribmap. For netCDF, LATS (optionally) outputs data conforming to the COARDS standard and again readible by GrADS.

LATS is a programming

When netCDF is not really netCDF

The support of "SDF" formats in GrADS, implemented by Don Hooper of CDC, uses the HDF library which reads both UNIDATA netCDF (the real netCDF) and HDF SDS (HDF Scientific Data Set) data conforming to the COARDS convention for gridded data. While this is a reasonable approach for reads it presented a real dilema when I added the LATS write capability.

"Would the real netCDF please stand up?"

The write functions in the Don's libraries, although they function just like the UNIDATA netCDF libraries, actually output HDF!!!

Support for 365 day calendars

The key word 365_day_calendar in the options line of the .ctl file will put the GrADS "time model" (how time and grid coordinates are related) into a no leap year mode. This feature supports the pentad data of NCEP's CPC, i.e., each year contains 73 pentads starting at the same day of the year (e.g., 3 january). During leap years the CPC pentad containing 29 February becomes a hexad(?) (6 day average).

CAUTION!!! Like many things in GrADS, we assume you all are big boys and girls when it comes to knowing your data. That is, GrADS gives you the rope to hang yourself in exchange for more control over the data and plots. This feature can be very dangerous if not used carefully as it changes the fundamental relationship between "world" and "grid" time.

With that said, I did place a restriction on this feature -- only one time model can be in effect during a GrADS session and it is set by the first data set opened. For example, if the you open a 365-day calendar data set and then try to open another with the standard Gregorian calendar (the default), GrADS will refuse and give you a warning. Similarily for opening standard then 365-day files. The fundamental problem is that only one world (real) time can exist at a given time.

A Climatology Convention?

To support the "climatology" calendars in LATS I have defined data sets with <= 12 months representative of climatology (or the annual cycle) to start in the year 2. Why not 1? That's because CDC uses this for their climatology and fixed fields (e.g., land-sea masks). Our convention at PCMDI is to use 1 for fixed fields and 2 for climatology.

This does NOT mean that climatology files are internally set to "seasonal." Rather, this is ONLY a proposed convention (the year can be changed in the LATS code). The user must still explicitly set up data to be treated as independent of year (and/or month) using

modify GGGGG seasonal|diurnal

where GGGGG is a defined grid.

New Display Projections...

Close File Command

You can now close the last file open using the

close ###

where ### is the file number (the last one only)

Various Bug Fixes

set grid horizontal for time on x axis
this failed and produced grid lines in BOTH x and y.

Variable Naming / Description Restrictions in the Data Descriptor (.ctl) file Relaxed

Previous versions set a limit of 8 characters and no _ in the variable names in the .ctl file. Further, the description was truncated at 79 characters causing problems for very long descriptions and parsing in gribmap. For example, the variable description in this .ctl file:

vars 1
my_variable 0 1 this is a super duper very long and tedious description of the variable which is actauly the "variable name in GrADS"
endvars

would not work properly. Now, the restriction is the variable name can be up to 15 characters long and can include the underscore charcter (_).

New Statistics from gxout stat

Listing station values using set stnprint on

The print command is the same as say in the GrADS scripting language

As a matter of personal convenience when writing perl and grads scripts together, I added the script command print which is identical to say.l Thus, the syntax of printing in both systems is more similar. Ditto for comments.

New Comment Character: #

The * character in the first column of .ctl files and scripts is used to tell GrADS the line is a comment. When writing perl and grads scripts together I got tired of having to use * in grads and # in perl, so I changed grads to support both * and #

Modifying the Value of Defined Variables

A previously undocumented feature of 1.5.1 is the ability to change the value of defined variables. A typical application might be to correct the value of a bad grid point discoverd through a maskout operation. Here is a typical sequence of events in a script. I'll use the model.ctl file in the grd_data directory of sample data available at ftp://sprite.llnl.gov/pub/fiorino/grads/data/grads.test.data.tar.

'open model.ctl'
*
*  set the dimension env to a sub area of this globla
*
'set lat 0 90'
'set lon 180 360'
'p=psl/100'
*
*  you must display before finding a value
*
'd p'
*
*  use the mouse to find a value to change
*
'q pos'
xscreen=subwrd(result,3)
yscreen=subwrd(result,4)
*
*  convert to grid coordinates
*
'q xy2gr 'xscreen' 'yscreen
xgrid=subwrd(result,3)
ygrid=subwrd(result,6)
*
* get the value of the defined variable p
*
'q defval p 'xgrid' 'ygrid
pval=subwrd(result,3)
*
* the value is 992.81;
* change to 965
*
pnewval=965
'set defval p 'xgrid' 'ygrid' 'pnewval
*
*  display the new field
*
'd p'

Changes to the code...

I confined my changes to a few routines and headers and new code was put in separate files

Program Comment
grads.c added -help, initialization of the galats struct
gauser.c set and query for the LATS interface
gagx.c LATS output like fwrite
gaddes.c input of new gribmap format (supports both version 1 and 2), set of the calendar, more robust .ctl parsing
gribmap.c this just does command processing and passes data to the actual gribmapper in the new code gagmap.c
gribscan
gxps
gxtran
stnmap
New Code (.c, .h, .sh) Description
gagmap.c the routines that do the actual mapping so that gribmaps can be created outside gribmap
galats.c The GrADS-LATS interface routines
gagmap.h global variables passed from the interface routine
gxeps.c Matt Muennich GrADS meta->eps convert utility now part of standard distribution
wgrib.c Wesley Ebisuzaki's GRIB decoder/processor
lats.c
latsint.c
latsgrib.c
latsgribmap.c
latsnc.c
latsstat.c
latstime.c
fgbds.c Fiorino GRIB (fgrib) encoder BDS maker
fgutil.c Fiorino GRIB (fgrib) encoder utilities
lats.h
latsint.h
latsparm.h
latstime.h
fgrib.h fgrib user interface
fgrib_init.h internal structs for fgrib


Send Comments to Mike Fiorino fiorino@llnl.gov, PCMDI, LLNL

Last updated 23 January, 1997
LLNL Disclaimers
UCRL-MI-NNNNN