|
|
|
Remote access using Matlab | Doc home | Madrigal home |
This page describes the remote Matlab API, and gives two examples of using this API. The first example uses all the basic methods, and outputs text. The second example creates a simple pcolor plot.
The remote Matlab API is organized in the same way as the Madrigal data model, from Instrument at the highest level, down to the level of data values. Readers who are not familiar with the Madrigal data model should read the material in that section before proceeding with this tutorial.
This API and example have been tested on both Windows and Linux, and require only access to the internet and Matlab 5 or greater to run. It is available for download here.
The following are methods specific to atmospheric science. These methods all use the matlab methods above:
***********************************************************
getMadrigalCgiUrl parse the main madrigal page to get the cgi url
The calling syntax is:
[cgiurl] = getMadrigalCgiUrl(url)
where url is the url of the Madrigal home page. For example,
getMadrigalCgiUrl('http://www.haystack.mit.edu/madrigal/') would
return 'http://www.haystack.mit.edu/cgi-bin/madrigal/'
***********************************************************
getInstrumentsWeb returns an array of instrument structs of instruments found on remote Madrigal server.
inputs: cgiurl (string) to Madrigal site cgi directory
(Example: 'http://www.haystack.mit.edu/cgi-bin/madrigal/')
Note that method getMadrigalCgiUrl converts homepage url into cgiurl.
output:
instArray - array of instrument structs found
instrument struct has the fields:
instrument.name (string) Example: 'Millstone Hill Incoherent Scatter Radar'
instrument.code (int) Example: 30
instrument.mnemonic (3 char string) Example: 'mlh'
instrument.latitude (double) Example: 45.0
instrument.longitude (double) Example: 110.0
instrument.altitude (double) Example: 0.015 (km)
Raises error if unable to return instrument array.
Example: getInstrumentsWeb('http://www.haystack.mit.edu/cgi-bin/madrigal/')
***********************************************************
getExperimentsWeb returns an array of experiment structs given input filter arguments from a remote Madrigal server.
Inputs:
1. cgiurl (string) to Madrigal site cgi directory
(Example: 'http://www.haystack.mit.edu/cgi-bin/madrigal/')
Note that method getMadrigalCgiUrl converts homepage url into cgiurl.
2. instCodeArray - a 1 X N array of ints containing selected instrument codes. Special value of 0 selects all instruments.
3. starttime - Matlab datenum double (must be UTC)
4. endtime - Matlab datenum double (must be UTC)
5. localFlag - 1 if local experiments only, 0 if all experiments
Return array of Experiment struct (May be empty):
experiment.id (int) Example: 10000111
experiment.url (string) Example: 'http://www.haystack.mit.edu/cgi-bin/madtoc/1997/mlh/03dec97'
experiment.name (string) Example: 'Wide Latitude Substorm Study'
experiment.siteid (int) Example: 1
experiment.sitename (string) Example: 'Millstone Hill Observatory'
experiment.instcode (int) Code of instrument. Example: 30
experiment.instname (string) Instrument name. Example: 'Millstone Hill Incoherent Scatter Radar'
experiment.starttime (double) Matlab datenum of experiment start
experiment.endtime (double) Matlab datenum of experiment end
experiment.isLocal (int) 1 if local, 0 if not
Raises error if unable to return experiment array
Example: expArray = getExperimentsWeb('http://www.haystack.mit.edu/cgi-bin/madrigal/', ...
30, datenum('01/01/1998'), datenum('12/31/1998'), 1);
***********************************************************
getCgiurlForExperiment returns cgiurl of experiment struct as returned by getExperimentsWeb or getExperiments.
inputs: experiment struct as returned by getExperimentsWeb or getExperiments.
output:
cgiurl of experiment
Simply truncates experiment.url to remove /madtoc/<YYYY>/<inst>/<date>
Example: If expArray is the value returned in the getExperimentsWeb example, and
expArray(1).url = 'http://madrigal.haystack.mit.edu/cgi-bin/madrigal/madtoc/1998/mlh/07jan98', then
getCgiurlForExperiment(expArray(1))
returns:
'http://madrigal.haystack.mit.edu/cgi-bin/madrigal/'
***********************************************************
getExperimentFilesWeb returns an array of experiment file structs given experiment id from a remote Madrigal server.
Note that it is assumed that experiment is local to cgiurl. If not,
empty list will be returned.
Inputs:
1. cgiurl (string) to Madrigal site cgi directory that has that
experiment.
(Example: 'http://www.haystack.mit.edu/cgi-bin/madrigal/')
Note that method getCgiurlForExperiment returns cgiurl for a given experiment struct.
2. experiment id (int) as returned by getExperiments or
getExperimentsWeb
Return array of Experiment File struct (May be empty):
file.name (string) Example '/opt/mdarigal/blah/mlh980120g.001'
file.kindat (int) Kindat code. Example: 3001
file.kindatdesc (string) Kindat description: Example 'Basic Derived Parameters'
file.category (int) (1=default, 2=variant, 3=history, 4=real-time)
file.status (string)('preliminary', 'final', or any other description)
file.permission (int) 0 for public, 1 for private
Raises error if unable to return experiment file array
Example: expFileArray = getExperimentFilesWeb('http://www.haystack.mit.edu/cgi-bin/madrigal/', 10001686);
***********************************************************
getParametersWeb returns an array of parameter structs given filename from a remote Madrigal server.
Note that it is assumed that filename is local to cgiurl. If not,
empty list will be returned.
Inputs:
1. cgiurl (string) to Madrigal site cgi directory that has that
filename.
(Example: 'http://www.haystack.mit.edu/cgi-bin/madrigal/')
Note that method getMadrigalCgiUrl converts homepage url into cgiurl.
2. filename (string) as returned by getExperimentFiles or
getExperimentFilesWeb
Return array of Parameter struct:
parameter.mnemonic (string) Example 'dti'
parameter.description (string) Example:
"F10.7 Multiday average observed (Ott) - Units: W/m2/Hz"
parameter.isError (int) 1 if error parameter, 0 if not
parameter.units (string) Example "W/m2/Hz"
parameter.isMeasured (int) 1 if measured, 0 if derivable
parameter.category (string) Example: "Time Related Parameter"
parameter.isSure (int) 1 if can be found for all records, 0 if only
for some records (implies not all records have same measured
parameters)
Raises error if unable to return parameter array
Example: parmArray = getParametersWeb('http://www.haystack.mit.edu/cgi-bin/madrigal/', ...
'/opt/madrigal/experiments/1998/mlh/07jan98/mil980107g.001')
***********************************************************
isprintWeb Create an isprint-like 3D array of doubles via
a command similar to the isprint command-line
application, but access data via the web. This is the command
that actually gets the measured data from a given Madrigal file.
Parameters that can be derived from those in the file are also
available via this file. See http://www.haystack.mit.edu/madrigal/ug_commandLine.html
for details.
The calling syntax is:
[records] = isprintWeb(cgiurl, file, parms, user_fullname, user_email, user_affiliation, [filters, [missing, [assumed, [knownbad] ] ] ])
where
cgiurl (string) to Madrigal site cgi directory that has that
filename.
(Example: 'http://www.haystack.mit.edu/cgi-bin/madrigal/')
Note that method getMadrigalCgiUrl converts homepage url into cgiurl.
file is path to file
(example = '/home/brideout/data/mlh980120g.001')
parms is the desired parameters in the form of a comma-delimited
string of Madrigal mnemonics (example = 'gdlat,ti,dti')
user_fullname - is user name (string)
user_email - is user email address (string)
user_affiliation - is user affiliation (string)
filters is the optional filters requested in exactly the form given in isprint
command line (example = 'time1=15:00:00 date1=01/20/1998
time2=15:30:00 date2=01/20/1998 filter=ti,500,1000')
See: http://www.haystack.mit.edu/madrigal/ug_commandLine.html for details
missing is an optional double to represent missing values. Defaults to NaN
assumed is an optional double to represent assumed values. Defaults to NaN
knownbad is an optional double to represent knownbad values. Defaults to NaN
The returned records is a three dimensional array of double with the dimensions:
[Number of rows, number of parameters requested, number of records]
If error or no data returned, will return error explanation string instead.
Example: data = isprintWeb('http://www.haystack.mit.edu/cgi-bin/madrigal/', ...
'/opt/madrigal/experiments/1998/mlh/07jan98/mil980107g.001', ...
'gdlat,ti,dti', ...
'Bill Rideout', 'wrideout@haystack.mit.edu', 'MIT');
***********************************************************
madCalculatorWeb Create a matrix of doubles via a the Madrigal derivation engine for a time and range of lat, long, and alt
The calling syntax is:
[record] = madCalculatorWeb(cgiurl, time, startLat, endLat, stepLat, startLong, endLong, stepLong,
startAlt, endAlt, stepAlt, parms)
where
cgiurl (string) to Madrigal site cgi directory that has that
filename.
(Example: 'http://www.haystack.mit.edu/cgi-bin/madrigal/')
Note that method getMadrigalCgiUrl converts homepage url into cgiurl.
time - Matlab datenum double (must be UTC) 7. startLat - Starting geodetic latitude, -90 to 90 (required)
endLat - Ending geodetic latitude, -90 to 90 (required)
stepLat - Latitude step (0.1 to 90) (required)
startLong - Starting geodetic longitude, -180 to 180 (required)
endLong - Ending geodetic longitude, -180 to 180 (required)
stepLong - Longitude step (0.1 to 180) (required)
startAlt - Starting geodetic altitude, >= 0 (required)
endAlt - Ending geodetic altitude, > 0 (required)
stepAlt - Altitude step (>= 0.1) (required)
parms is the desired parameters in the form of a comma-delimited
string of Madrigal mnemonics (example = 'gdlat,ti,dti')
The returned record is a matrix of doubles with the dimensions:
[(num lat steps * num long steps * num alt steps), 3 + num of parms]
The first three columns will always be lat, long, and alt, so there are three
additional columns to the number of parameters requested via the parms argument.
If error or no data returned, will return error explanation string instead.
Example: result = madCalculatorWeb('http://www.haystack.mit.edu/cgi-bin/madrigal', ...
now,45,55,5,45,55,5,200,300,50,'bmag,bn');
***********************************************************
getConjugatePoint returns the magnetic conjugate lat and long for a given
gdlat, glon, gdalt, along with the corrected geomagnetic lat and long of
the starting point.
Inputs:
madrigalUrl: home page of madrigal site to use. Example:
'http://www.haystack.mit.edu/madrigal' (Not cgi url)
gdlat, glon, gdalt - specify point in geodetic terms
year, month, day, hour, min, sec - specify time
Returns:
magconjlat - geodetic latitude of magnetic conjugate point
magconjlon - geodetic longitude of magnetic conjugate point
cgm_lat - Corrected geomagnetic latitude of input point
cgm_long - Corrected geomagnetic longitude of input point
Uses madCalculatorWeb
Example: [magconjlat, ...
magconjlon, ...
cgm_lat, ...
cgm_long] = getConjugatePoint('http://www.haystack.mit.edu/madrigal', ...
42, -70, 1000, ...
1998,1,1,0,0,0);
***********************************************************
getGsmPoint returns a single point in the GSM XY plane via Tsyganenko
field for given time, gdlat, glon, gdalt
Inputs:
madrigalUrl: home page of madrigal site to use. Example:
'http://www.haystack.mit.edu/madrigal' (Not cgi url)
gdlat, glon, gdalt - specify point in geodetic terms
year, month, day, hour, min, sec - specify time
Returns:
[xgsm, ygsm] - point in GSM XY plane (magnetic equatorial plane)
where Tsyganenko field line crosses.
Uses madCalculatorWeb
Example: [xgsm, ygsm] = getGsmPoint('http://www.haystack.mit.edu/madrigal', ...
42, -70, 1000, ...
1998,1,1,0,0,0);
***********************************************************
getIonosphericTerminator returns an array of lat, lon defining the
ionoshperic terminator at the given time and altitude
Inputs:
madrigalUrl: home page of madrigal site to use. Example:
'http://www.haystack.mit.edu/madrigal'
time - in form '15-Oct-2002 12:54:00'
alt - altitude in km at which to find the terminator
gridSize = size of grid in degrees to define the terminator
Returns:
[term] - an n by 2 array of lat, longitude pairs defining the terminator.
Algorithm: Madrigal will calculate shadowHeight for each point on a global grid
at the given time. Each latitude will be scanned, and any point where the change in
shadowheight crosses the given alt will be added to term array.
Uses madCalculatorWeb
Example: result = getIonosphericTerminator('http://www.haystack.mit.edu/madrigal', ...
'15-Oct-2002 12:54:00', ...
1000, 1);
***********************************************************
getMagnetopause returns a matrix of xgsm, ygsm points on the
sun-facing side of the magnetopause
This method will simply call madCalculatorWeb for a series of latitude steps.
Inputs:
madrigalUrl: home page of madrigal site to use. Example:
'http://www.haystack.mit.edu/madrigal'
year, month, day, hour, min, sec - specify time
Returns:
An (n x 2) matrix of (xGsm, yGsm) points on the GSM equatorial plane that
define the magnetopause.
Uses madCalculatorWeb to calculate tsyg_eq_xgsm and tsyg_eq_ygsm for a
grid of lat and long.
This method searches for the maximum xgsm in the 14 ygsm bins
(-7< ygsm < -6), (-6< ygsm < -5),...(6< ygsm < 7)
Example: magnetopause = getMagnetopause('http://www.haystack.mit.edu/madrigal', ...
1998,1,1,0,0,0);
***********************************************************
mapGeodeticToGsm returns a 2 x N array of GSM X and Y where a field
line intersects the input points, and is followed to the GSM Z=0 plane.
Uses Tsyganenko 2001 field model
Inputs:
madrigalUrl - url to madrigal site, such as
'http://www.haystack.mit.edu/madrigal'
year, month, day, hour, min, sec - time of interest
latArr - length N array of input latitude values
lonArr - length N array of input longitude values (length = latArr)
altArr - length N array of input altitude values (length = latArr)
Returns a 2 X N matrix where column 1 is X GSM, column 2 is Y GSM, and N is
the length of the input matrices. Z GSM is always 0. If no solution for a
given input point, lat and long will be NaN.
Example:
res = mapGeodeticToGsm('http://www.haystack.mit.edu/madrigal', ...
2004,1,1,0,0,0, ...
[30 35 40 45 50],[270 270 270 270 270], ...
[1000 1000 1000 1000 1000]);
***********************************************************
mapGsmToAltitude returns a 2 x N array of geodetic lat and long where a field
line intersects a given stop altitude for arrays of starting points in GSM
coordinates.
Inputs:
madrigalUrl - url to madrigal site, such as
'http://www.haystack.mit.edu/madrigal'
model - 0 for Tsyganenko, 1 for IGRF
stopAlt - altitude in km to intersect with
year, month, day, hour, min, sec - time of interest
xgsmArr - array of input xgsm values
ygsmArr - array of input ygsm values (length = xgsmArr)
zgsmArr - array of input zgsm values (length = xgsmArr)
northHemisphereFlag: 1, map into northern hemisphere, 0
map into southern hemisphere
Returns a 2 X N matrix where column 1 is lat, column 2 is long, and N is
the length of the input matrices. The end altitude is always the
stopAlt. If no solution for a given input point, lat and long will be
NaN.
Example:
res = mapGsmToAltitude('http://www.haystack.mit.edu/madrigal', ...
0,1000,2004,1,1,0,0,0, ...
[0.269 0.3 0.3 0.3 0.3],[0.583 0.6 0.6 0.6 0.6], ...
[-0.95987 -10.0 -1.0 -10.0 -1.0],1);
***********************************************************
Two examples are given below. The first example uses all the methods, and outputs text. The second example creates a simple pcolor plot.
Simple Example - text output
This simple example Matlab script uses most of the methods discussed above.
% demo program of madmatlab running on a pc or linux
% first, get url of installation to test
madurl = input('Enter the url of the home page of the Madrigal site to test:', 's');
cgiurl = getMadrigalCgiUrl(madurl)
'List all instruments, and their latitudes and longitudes:'
instArray = getInstrumentsWeb(cgiurl);
for i = 1:length(instArray)
[s,errmsg] = sprintf('Instrument: %s, at lat %f and long %f', ...
instArray(i).name, ...
instArray(i).latitude, ...
instArray(i).longitude);
s
end
% now list all experiments from local Madrigal site with mlh (code 30) in
% 1998 - should be data if default files installed...
startdate = datenum('01/01/1998');
enddate = datenum('12/31/1998');
expArray = getExperimentsWeb(cgiurl, 30, startdate, enddate, 1);
for i = 1:length(expArray)
[s,errmsg] = sprintf('Experiment name: %s, at url %s', ...
expArray(i).name, ...
expArray(i).url);
s
end
% now list all files in the first experiment
expFileArray = getExperimentFilesWeb(cgiurl, expArray(1).id);
for i = 1:length(expFileArray)
[s,errmsg] = sprintf('File name: %s, with kindat %i', ...
expFileArray(i).name, ...
expFileArray(i).kindat);
s
end
% now first 2 parameters in the last file
parmArray = getParametersWeb(cgiurl, expFileArray(end).name)
for i = 1:10
[s,errmsg] = sprintf('Parameter mnemonic: %s, description "%s" -- isMeasured = %i', ...
parmArray(i).mnemonic, ...
parmArray(i).description, ...
parmArray(i).isMeasured);
s
end
% finally, run isprint for that file for first two parameters
parmStr = sprintf('%s,%s', parmArray(1).mnemonic, parmArray(2).mnemonic);
data = isprintWeb(cgiurl, expFileArray(1).name, parmStr, 'Bill Rideout', 'wrideout@haystack.mit.edu', 'MIT');
% print first 10 records
data(:,:,1:10)
Plotting Example
This example creates a simple pcolor plot.
% this example plots Ti versus altitude and time as a pcolor plot
%
% To turn this script into a function, replace the hard-coded filename
% with an input argument
%
% Before running this script, install the Remote Matlab Madrigal API
% available from http://www.openmadrigal.org
%
% Complete documentation is available at
% http://www.haystack.edu/madrigal/remoteMatlabAPI.html
filename = '/usr/local/madroot/experiments/2003/tro/05jun03/NCAR_2003-06-05_tau2pl_60_uhf.bin';
eiscat_cgi_url = 'http://www.eiscat.se/madrigal/cgi-bin/';
% download the following parameters from the above file: ut, gdalt, ti
parms = 'ut,gdalt,ti';
filterStr = 'filter=gdalt,200,600 filter=ti,0,5000';
% returns a three dimensional array of double with the dimensions:
%
% [Number of rows, number of parameters requested, number of records]
%
% If error or no data returned, will return error explanation string instead.
data = isprintWeb(eiscat_cgi_url, filename, parms, 'Bill Rideout', 'brideout@haystack.mit.edu', 'MIT', filterStr);
% thats it for Madrigal - the rest of this example is purely a plotting
% demo
% get size of returned data
sizeData = size(data);
% create X, Y, Ti arrays for pcolor
X = zeros(sizeData(1), sizeData(3), 'double');
Y = zeros(sizeData(1), sizeData(3), 'double');
Ti = zeros(sizeData(1), sizeData(3), 'double');
% loop through alts and times
for alt = 1:sizeData(1)
for time = 1:sizeData(3)
X(alt,time) = data(alt,1,time);
Y(alt,time) = data(alt,2,time);
Ti(alt,time) = data(alt,3,time);
end
end
pcolor(X,Y,Ti);
shading flat;
caxis([800, 2000]);
cb = colorbar;
|
|
|
Remote access using Matlab | Doc home | Madrigal home |