Data Sources
Updated: 18 July 2003
There are three
DODS servers for accessing the data directly over the internet from your
model, visualization software, or other programs:
Additionally, data are now being made available by FTP for those who
cannot access any of the servers above because of firewall, compiler, or
other issues. Access to the FTP server is tightly controlled, and should
be use as a
last resort only if you have tried and cannot access the data sets interactively from your programs (see the section below on
GSWP-2 and DODS
for guidelines on using the DODS-enabled libraries for your favorite programming
language or applications). For information on accessing the FTP site, see
the bottom of the page.
GSWP-2 and DODS
The final input data sets for the participating GSWP-2 LSSs will be served
over the Internet via several Distributed Oceanographic Data System (
DODS)
servers. DODS provides a protocol for remotely accessing and subsetting
large data sets (see:
http://www.unidata.ucar.edu/packages/dods/).
DODS-enabled clients exist via the NetCDF protocol for FORTRAN and C, and
a growing number of popular graphics and analysis packages are DODS-capable.
In a nutshell, the “open” statement, which typically accesses a file on local
disk via its pathname, instead accesses the DODS server via its URL using
HTTP addressing. DODS includes metadata, so that served data sets are
self-describing. DODS reduces strains on local disk resources, and,
when properly used, is not taxing on the network or the server hardware.
In order to reduce trans-oceanic networking delays, we anticipate serving
the data from multiple DODS servers, operating in mirror mode, in North America
(COLA, USA), Asia (University of Tokyo, Japan), and Europe (IPSL, France).
If this approach is intractable to one or more modelers, we will consider
alternate solutions, probably involving shipping of data on magnetic media.
The GrADS-DODS server (GDS) software, in use at the North American and Asian data servers,
allows for dual-mode access to compressed NetCDF data in the CF convention
(including compression by gathering) — either as the original one-dimensional spatial
vector of data (for access by LSSs), or as a repopulated grid (for easy display
purposes). All DODS functionality is available through GDS, as well
as the ability to encapsulate computational expressions and commands which
direct the data server to perform calculations (using the GrADS computing
engine), and return only the results to the requesting client. More
information on the GrADS-DODS server is available at the
GrADS web site.
The GrADS-DODS server (GDS) software also allows you to access the complete
time series of any variable as if it were all in one continuous file. In
fact, there is a separate file for each month for both the monthly and 3-hourly
data sets. This preponderance of files is hidden from the GDS user because
of GrADS' templating ability, using time information in the directoy and
file names to concatenate separate data files into a continuous stream.
Each of the GSWP-2 DODS data repositories will have a different URL. Users
should choose the server closest to their location for optimal network performance.
The URLs of the GSWP-2 DODS servers are:
For example, to access the land-only (compressed by gathering) SoilClass
file in North America with a DODS-enabled client, the path would be:
http://www.monsoondata.org:9090/dods/gswp/vector/fixed/SoilClass
A catalog of the data sets on the DODS server are available by accessing the directories from a web browser:
http://www.monsoondata.org:9090/dods/gswp/vector/
http://www.monsoondata.org:9090/dods/gswp/vector/fixed/
http://www.monsoondata.org:9090/dods/gswp/grid/
http://www.monsoondata.org:9090/dods/gswp/grid/fixed/
The catalogs list the available variable files and links to several documentation
files for each variable files, including a web-formatted plain-language description
of the file and metadata called
info, a DODS data descriptor structure file called
dds, and a NetCDF-style data attribute structure file called
das.
The European server is a conventional DODS
server at IPSL. The directory structure is the same as on the North American
server, except that the multi-file nature of the time-varying fields is now
clearly evident and requires you to access each month by a separate address.
Direct access to the data using DODS-enabled software is through the URL
path:
http://dods.ipsl.jussieu.fr/cgi-bin/nph-dods//gswp/
Because the European server is a conventional DODS server, you can view the
data files directly from the web. Data sets may be downloaded using the
Lynx browser, or using the
wget command:
wget -r http://dods.ipsl.jussieu.fr/gswp/
However, only the vector (compressed by gathering) files are accessable using
the conventional DODS server. The repopulated grids are not accessible from
the European site.
There are four directories for the GSWP-2 forcing data. The one-dimensional land-only data are in the
vector directory, and the fully-populated two-dimensional global grids are in the
grid
directory (with water and Antarctic points set to the missing data value).
The time-invariant fields are in two subdirectories called
fixed. The files in the
vector directory are smaller and will download more quickly. The NetCDF metadata, when combined with the file
gswp/grid/fixed/Landmask,
have all of the information necessary to navigate the compressed data vectors.
So it is advisable to run the land surface models using this compressed
data, rather than downloading the complete 2-D grids and masking out the
missing data points. The 2-D grids are provided for quick visualization
or analysis applications. There is an additional directory called
field which contains NetCDF formatted versions of several observational data sets, for validation or calibration purposes.
Below are listed the GSWP-2 input data fields by thier ALMA variable name,
and the path to access each one on the DODS servers.
|
Fixed Fields
|
File: gswp/vector/ or gswp/grid/
|
|
SoilClass
|
fixed/soilclass
|
|
SoilDepth
|
fixed/Soildepth
|
|
Clay
|
fixed/clay
|
|
Sand
|
fixed/sand
|
|
Silt
|
fixed/silt
|
|
Organic
|
fixed/organic
|
|
Elevation
|
fixed/elevation
|
|
Slope
|
fixed/slope
|
|
W_fieldcap
|
fixed/w_fieldcap
|
|
W_wilt
|
fixed/w_wilt
|
|
W_sat
|
fixed/w_sat, fixed/w_sat_cea84
|
|
W_bpower
|
fixed/w_bpower_cea84
|
|
W_sat_hydc
|
fixed/w_sat_hydc, fixed/w_sat_hydc_cea84
|
|
W_sat_matp
|
fixed/w_sat_matp_cea84
|
|
Albedo_vi
|
fixed/albedo_vi
|
|
Albedo_ir
|
fixed/albedo_ir
|
|
Albedo_soil
|
fixed/albedo_soil
|
|
VegClass
|
fixed/vegclass_igbp, fixed/vegclass_sib, fixed/vegclass_bats
|
|
classFrac
|
fixed/classfrac
|
|
RootDepth
|
fixed/rootdepth_50,
fixed/rootdepth_95
|
|
vegFrac
|
fixed/vegfrac_uk
|
|
Cti
|
fixed/cti
|
|
Grid only
|
File: gswp/grid/
|
|
Landmask
|
fixed/landmask
|
|
Monthly Fields
|
File: gswp/vector/ or gswp/grid/
|
|
LAI
|
lai_uk
|
|
grnFrac
|
grnfrac_uk
|
|
NDVI
|
ndvi_uk
|
|
FPAR
|
fpar_uk
|
|
Z0Surf
|
z0surf_uk
|
|
DisplH
|
displh_uk
|
|
Albedo
|
albedo_csu
|
|
3-hourly Fields
|
File: gswp/vector/ or gswp/grid/
|
|
Tair
|
tair_cru
|
|
Qair
|
qair_cru
|
|
Wind
|
wind_ncep
|
|
SWdown
|
swdown_srb
|
|
LWdown
|
lwdown_srb
|
|
Psurf
|
psurf_ecor
|
|
Rainf
|
rainf_gswp
|
|
Rainf_C
|
rainf_c_gswp
|
|
Snowf
|
snowf_gswp
|