previous up next Installing Madrigal for the first time Doc home Madrigal home
Previous: Is Madrigal appropriate?   Up: Madrigal admin guide   Next: Upgrading Madrigal

Installing Madrigal for the first time

Prerequisites

Most of the prerequisites for Madrigal are pre-installed in any modern unix distribution. They are:

  1. A C, C++, and a FORTRAN compiler. The free GNU compilers may be downloaded from the GNU Website .
  2. tclsh, which may be downloaded from the Tcl Resource Center and libpng available from http://www.libpng.org/pub/png/libpng.html. Both these are typically included on unix distributions.
  3. A web server.
  4. The environmental variable MADROOT should be set to the path to the Madrigal installation directory. Its recommended this be done in your login script (eg, .bash_profile) because you will need it to run administrative scripts later.
  5. The environmental variable LD_LIBRARY_PATH should be set to $LD_LIBRARY_PATH:$MADROOT/lib. Its recommended this be done in your login script (eg, .bash_profile) because you will need it to run administrative scripts later.

The installation script will stop and issue a warning if any prerequisite is missing.

 

Installation instructions

Madrigal can be installed on any unix server with a web server. If you want to link your data in with data on other Madrigal servers, please notify the OpenMadrigal administrator. In general you do not need root permission to install madrigal once the prerequisites listed above are installed. You may need to be root to create the html and cgi directories in the web server. It is generally easiest to then chown for those two directories to the user that will be installing Madrigal.

  1. Create a directory to be used as your Madrigal root directory on your unix server. This directory will be referred to as MADROOT.
  2. Create the environment variable MADROOT with that directory path.
  3. Download the latest Madrigal distribution file, madrigal*.tar.gz from the OpenMadrigal distribution page to MADROOT.
  4. gunzip madrigal*.tar.gz
  5. tar -xf madrigal*.tar
  6. Repeat the 3 steps above to get and untar the Sample Experiments file experiments.tar.gz.
  7. Create two virtual directories on your webserver for your madrigal cgi files (which must allow scripts to run) and for your madrigal html files.
  8. In madroot, there will now be a file called madrigal.cfg.template. Copy it to madrigal.cfg, and edit all the configuration parameters, as described in the Editing madrigal.cfg section below.
  9. Edit the file MADROOT/metadata/siteTab.txt with a unique id to include your site. The format of siteTab.txt is described here. If you want to be an official Madrigal site that other Madrigal sites share data with, request the OpenMadrigal administrator to assign you this id. If this is just a test site, use any id not already in the siteTab.txt file. If you do not want all the sites in siteTab.txt included in your combined inventory listing, just remove these sites from siteTab.txt. Links to data from sites you leave in the siteTab.txt file will appear in your Madrigal web site.
  10. With Madrigal 2.6, users can now download cached HDF5 files. These cached HDF5 files are created during installation. You will want to make sure you have enough room in your experiments directory (typically about double the existing space used). You can create an ini file to modify how these HDF5 files are created. Instructions below describe how to added derived parameters and/or additional layouts for an given instrument, and optionally by kindat.
  11. Be sure to cd to MADROOT before running the following step. Then you should be able to complete the installation simply by typing 
    bash installMadrigal  &> install.log &
    There may be a long pause when running updateMaster near the end of the installation since the instParmTab.txt metadata file is being built for the first time by examining every data file, but future calls to updateMaster will be much faster since only new experiments are examined. Help with any installation errors is available from the OpenMadrigal administrator.
  12. If there were no errors, your madrigal installation should be running at the url given by madrigal. You can also test it by running MADROOT/bin/python testMadrigalBuild.py, which will test the ability to create plots.
  13. If you want to receive notices about updates to Madrigal, sign up for the openmadrigal-admin mailing list under www.openmadrigal.org.
  14. Set the script madroot/bin/updateMaster up as a cron job to run once a day.
  15. If you want to add any documentation pages specific to your site to the Madrigal documentation pages, see the other admin tasks section of this manual.
  16. If you want to add your own rules of the road to the Madrigal experiment page, see the other admin tasks section of this manual.

Editing the madrigal.cfg file

The madrigal.cfg file contains all the configuration information specific to your installation. This section discusses how to edit that file for each parameter. The madrigal.cfg.template file contains examples of each parameter.

Creating or modifying MADROOT/cachedFiles.ini

By default, Madrigal will create cached HDF5 versions of all default files when Madrigal 2.6 or later is installed. These cached HDF5 files are created so that the user will get a fast response when an HDF5 file is requested for download. Large HDF5 files can take minutes to create from the Cedar source file. By default, these cached HDF5 files only contain the parameters in the Cedar file, and arrange the data only in the default table layout. If you want these cached HDF5 files to have any additional derived parameters or extra layout formats, you will need to create a file called cachedFiles.ini in your MADROOT directory.

With Madrigal 2.6, there is only one additional layout available: 'array'. If you ask for this additional layout, your HDF5 file will contain a subdirectory called array, which will contain one or more array layouts. An array layout has separate data for each parameter, arranged in a grid of one or more spatial parameters and time. This array layout is defined by passing it a tuple of two lists: 1) a list of independent spatial parameters, and 2) a list of parameters used to split the data into separate arrays. This argument is described in more detail below.

When defining how you want you array to look, you need to consider two questions:

  1. What are the independent spatial parameters of my array of data?
  2. Does all the data in this file have the same values of the independent spatial parameters?

For example, for radar data, you would typically use range or altitude as the single independent spatial parameter. More than one independent spatial parameter is allowed; for example, with TEC maps the two independent spatial parameters are latitude (gdlat) and longitude (glon). An instrument capable of measuring volumetric data might have three independent spatial parameters.

Sometimes it makes sense to split a Madrigal file into separate arrays. For example, with phased arrays that measure multiple beam simultaneously, it would be helpful to have each beam be its own array. You can do this by adding the beamid parameter to the second list in the array setup argument. At Millstone, we would like to have separate arrays if a single pulse or alternating code is used (parameter mdtyp) or if different pulse lengths are used (pl parameter). So at Millstone Hill we pass in two parameters for our radar data: mdtyp and pl. A separate array is created for each unique combination of mdtyp and pl. If any combination contains no data, that array is skipped. The examples below should help to make this clear.

You can use different arguments for any combination of instrument and kindat. The cachedFile.ini file has section headings which are simply instrument ids (kinst code). The key value pairs can be:

  1. <kindat>_parms = <comma-delimited parameter list>
  2. <kindat>_formats = <python dictionary>
  3. default_parms = <comma-delimited parameter list>
  4. default_formats = <python dictionary>

If an instrument does not have a section, then defaults are used. If a given instrument has a default_parms or default_formats key, then that will be used, unless a key with the matching kindat (kind or data) is found. If an instrument has kindat keys and no default, and the kindat does not match, the default is used.

Here's three example cachedFiles.ini files.

In the first case, for kinst=30 (the Millstone ISR), kindats 13204 and 13300 are listed separately. If any other kindat is found, the default key is used. For default files, range is the independent spatial parameter, and the parameters mdtyp and pl are used to split the file into separate arrays:

# Millstone Hill radars 
[30]
# 13204 is F regions winds file

13204_parms =
13204_formats = {}
# 13300 is gridded ISR data
13300_parms = 
13300_formats = {}
# all other kindats should be as follows
default_parms = gdlat,glon,gdalt,ne,dne 
default_formats = {'array': (['range'], ['pl', 'mdtyp'])}

In the second case, for kinst=61 (the Poker Flat ISR), kindat 5960 is listed separately. For default files, range is the independent spatial parameter, and beamid is used to split the file into separate arrays:

# Poker Flat
[61]
# velocity vector is special case
5960_parms = 
5960_formats = {'array':'cgm_lat'}
default_parms = gdlat,glon,gdalt
default_formats = {'array':(['range'],['beamid'])}

In the third case, for kinst=8000 (GPS data), both gdlat and glon are the independent spatial parameters, and there is no splitting into separate arrays:

# gps data
[8000]
default_parms = gdlat,glon,gdalt
default_formats = {'array':(['gdlat', 'glon'],[])}
previous up next Installing Madrigal for the first time Doc home Madrigal home
Previous: Is Madrigal appropriate?   Up: Madrigal admin guide   Next: Upgrading Madrigal