MadrigalData is a class that acquires data from a particular Madrigal site.
Usage example:
import madrigalWeb.madrigalWeb
test = madrigalWeb.madrigalWeb.MadrigalData('http://madrigal.haystack.mit.edu/madrigal')
instList = test.getInstrumentList()
Non-standard Python modules used: None
Change history:
Written by Bill Rideout Feb. 10, 2004
Methods
|
|
|
|
__getSiteDict
|
__getSiteDict ( self )
__getSiteDict returns a dictionary with key = site id, value= site url.
Uses getMetadata cgi script
|
|
__getSiteId
|
__getSiteId ( self )
__getSiteId returns the local site id
Uses getMetadata cgi script
|
|
__init__
|
__init__ ( self, url )
__init__ initializes a MadrigalData object.
Inputs:
url - (string) url of main page of madrigal site. Example: 'http://madrigal.haystack.mit.edu/madrigal'
Affects: Converts main page to cgi url, and stores that.
Also stores self.siteDict, with key = site id, value= site url, and self.siteId
Exceptions: If url not found.
Exceptions
|
|
ValueError, 'invalid url: ' + str( url )
ValueError, 'unable to open url ' + str( url )
|
|
|
compareVersions
|
compareVersions (
self,
ver1,
ver2,
)
compareVersions returns -1 if ver1 < ver2, 0 if equal, 1 if ver1 > ver2
Inputs: version number strings, in form number dot number (any number of dots)
|
|
downloadFile
|
downloadFile (
self,
filename,
destination,
user_fullname,
user_email,
user_affiliation,
format='simple',
)
downloadFile will download a Cedar file in the specified format.
Inputs:
filename - The absolute filename to as returned via getExperimentFiles.
destination - where the file is to be stored
user_fullname - full name of user making request
user_email - email address of user making request
user_affiliation - affiliation of user making request
format - file format desired. May be simple , hdf5 , madrigal ,
blockedBinary , ncar ,
unblockedBinary , or ascii . Default is simple
Simple is a simple ascii space delimited column format.
Simple and hdf5 are recommended since they are standard formats
hdf5 format works for Madrigal 2.6 or later
Exceptions
|
|
IOError, 'downloadFile with hdf5 format requires Madrigal 2.6 or greater, but this site is version %s' %( self._madVers )
ValueError, 'Illegal format specified: %s' %(str( format ) )
|
|
|
geodeticToRadar
|
geodeticToRadar (
self,
slatgd,
slon,
saltgd,
gdlat,
glon,
gdalt,
)
geodeticToRadar converts arrays of points in space to az, el, and range.
Input arguments:
slatgd - radar geodetic latitude
slon - radar longitude
saltgd - radar altitude
gdlat - either a single geodetic latitude, or a list of geodetic latitudes
glon - either a single longitude, or a list of longitudes. If so, len(gdlat)
must = len(glon)
gdalt - either a single deodetic altitude, or a list of geodetic altitudes.
If so, len(gdalt) must = len(gdlat)
Returns:
A list of lists, where each list contains 3 floats (az, el, and range)
Exceptions
|
|
ValueError, 'No data found at url' + str( url )
ValueError, 'all lists most have same length'
ValueError, 'error raised using url ' + str( url ) + ' ' + str( page )
ValueError, 'unable to open url ' + str( url )
|
|
|
getAllInstruments
|
getAllInstruments ( self )
returns a list of all MadrigalInstruments at the given Madrigal site
Exceptions
|
|
ValueError, 'No data found at url' + str( url )
ValueError, 'error raised using url ' + str( url ) + ' ' + str( page )
ValueError, 'unable to open url ' + str( url )
|
|
|
getExperimentFileParameters
|
getExperimentFileParameters ( self, fullFilename )
getExperimentFileParameters returns a list of all measured and derivable parameters in file
Inputs:
fullFilename - full path to experiment file as returned by getExperimentFiles.
Outputs:
List of MadrigalParameter objects for that fullFilename. Includes both measured
and derivable parameters in file.
Exceptions
|
|
ValueError, 'error raised using url ' + str( url ) + ' ' + str( page )
ValueError, 'unable to open url ' + str( url )
|
|
|
getExperimentFiles
|
getExperimentFiles (
self,
id,
getNonDefault=False,
)
returns a list of all default MadrigalExperimentFiles for a given experiment id
Inputs:
id - Experiment id.
getNonDefault - if False (the default), only get default files, or realtime
files if no default files found. If True, get all files.
In general, users should set this to False because default files
are the most reliable.
Outputs:
List of MadrigalExperimentFile objects for that experiment id
Exceptions
|
|
ValueError, """Illegal experiment id -1. This is usually caused by calling
getExperiments with the isLocal flag set to 0. To get the experiment id for a non-local
experiment, you will need to create a new MadrigalData object with the url of the
non-local experiment (MadrigalExperiment.madrigalUrl), and then call
getExperiments a second time using that Madrigal url. This is because
while Madrigal sites share metadata about experiments, the real experiment ids are only
known by the individual Madrigal sites. See examples/exampleMadrigalWebServices.py
for an example of this.
"""
ValueError, 'error raised using url ' + str( url ) + ' ' + str( page )
ValueError, 'unable to open url ' + str( url )
|
|
|
getExperiments
|
getExperiments (
self,
code,
startyear,
startmonth,
startday,
starthour,
startmin,
startsec,
endyear,
endmonth,
endday,
endhour,
endmin,
endsec,
local=1,
)
returns a list of all MadrigalExperiments that meet criteria at the given Madrigal site
Inputs:
code - int or list of ints representing instrument code(s). Special value of 0 selects all instruments.
startyear - int or string convertable to int
startmonth - int or string convertable to int
startday - int or string convertable to int
starthour - int or string convertable to int
startmin - int or string convertable to int
startsec - int or string convertable to int
endyear - int or string convertable to int
endmonth - int or string convertable to int
endday - int or string convertable to int
endhour - int or string convertable to int
endmin - int or string convertable to int
endsec - int or string convertable to int
local - 0 if all sites desired, 1 (default) if only local experiments desired
Outputs:
List of MadrigalExperiment objects that meet the criteria. Note that if the returned
MadrigalExperiment is not local, the experiment id will be -1. This means that you
will need to create a new MadrigalData object with the url of the
non-local experiment (MadrigalExperiment.madrigalUrl), and then call
getExperiments a second time using that Madrigal url. This is because
while Madrigal sites share metadata about experiments, the real experiment ids are only
known by the individual Madrigal sites. See examples/exampleMadrigalWebServices.py
for an example of this.
Exceptions
|
|
ValueError, 'error raised using url ' + str( url ) + ' ' + str( page )
ValueError, 'unable to open url ' + str( url )
|
|
|
getVersion
|
getVersion ( self )
getVersion gets the version number of Madrigal in form number dot number etc.
Assumes version is 2.5 if no getVersionService.py installed
|
|
isprint
|
isprint (
self,
filename,
parms,
filters,
user_fullname,
user_email,
user_affiliation,
)
returns as a string the isprint output given filename, parms, filters without headers or summary.
Inputs:
filename - The absolute filename to be analyzed by isprint.
parms - Comma delimited string listing requested parameters (no spaces allowed).
filters - Space delimited string listing filters desired, as in isprint command
user_fullname - full name of user making request
user_email - email address of user making request
user_affiliation - affiliation of user making request
Returns:
a string holding the isprint output
Exceptions
|
|
ValueError, 'error raised using url ' + str( url )
ValueError, 'unable to open url ' + str( url )
|
|
|
listFileTimes
|
listFileTimes ( self, expDir=None )
listFileTimes lists the filenames and last modification times for all files in a Madrigal database.
Inputs: expDir - experiment directory to which to start. May be any directory or subdirectory below
experiments[0-9]*. Path may be absolute or relative to experiments[0-9]*. If None (the default),
include entire experiments[0-9]* directory(s). Examples: (/opt/madrigal/experiments/1998 ,
experiments/2002/gps )
Returns: a list of tuple of 1. filename relative to experiments[0-9]* directory, and 2) datetime in UT of
last file modification
Requires: Madrigal 2.6 or greater
Exceptions
|
|
IOError, 'listFileTimes requires Madrigal 2.6 or greater, but this site is version %s' %( self._madVers )
|
|
|
madCalculator
|
madCalculator (
self,
year,
month,
day,
hour,
min,
sec,
startLat,
endLat,
stepLat,
startLong,
endLong,
stepLong,
startAlt,
endAlt,
stepAlt,
parms,
oneDParmList=[],
oneDParmValues=[],
)
Input arguments:
year - int
month - int
day - int
hour - int
min - int
sec - int
startLat - Starting geodetic latitude, -90 to 90 (float)
endLat - Ending geodetic latitude, -90 to 90 (float)
stepLat - Latitude step (0.1 to 90) (float)
startLong - Starting geodetic longitude, -180 to 180 (float)
endLong - Ending geodetic longitude, -180 to 180 (float)
stepLong - Longitude step (0.1 to 180) (float)
startAlt - Starting geodetic altitude, >= 0 (float)
endAlt - Ending geodetic altitude, > 0 (float)
stepAlt - Altitude step (>= 0.1) (float)
parms - comma delimited string of Madrigal parameters desired
oneDParmList - a list of one-D parameters whose values should
be set for the calculation. Can be codes or mnemonics.
Defaults to empty list.
oneDParmValues - a list of values (doubles) associated with the one-D
parameters specified in oneDParmList. Defaults to empty list.
Returns:
A list of lists of doubles, where each list contains 3 + number of parameters doubles.
The first three doubles are the input latitude, longitude, and altitude. The rest of the
doubles are the values of each of the calculated values. If the value cannot be calculated,
it will be set to nan.
Example:
result = testData.madCalculator(1999,2,15,12,30,0,45,55,5,-170,-150,10,200,200,0,'bmag,bn')
result = [ [45.0, -170.0, 200.0, 4.1315700000000002e-05, 2.1013500000000001e-05]
[45.0, -160.0, 200.0, 4.2336899999999998e-05, 2.03685e-05]
[45.0, -150.0, 200.0, 4.3856400000000002e-05, 1.97411e-05]
[50.0, -170.0, 200.0, 4.3913599999999999e-05, 1.9639999999999998e-05]
[50.0, -160.0, 200.0, 4.4890099999999999e-05, 1.8870999999999999e-05]
[50.0, -150.0, 200.0, 4.6337800000000002e-05, 1.80077e-05]
[55.0, -170.0, 200.0, 4.6397899999999998e-05, 1.78115e-05]
[55.0, -160.0, 200.0, 4.7265400000000003e-05, 1.6932500000000001e-05]
[55.0, -150.0, 200.0, 4.85495e-05, 1.5865399999999999e-05] ]
Columns: gdlat glon gdalt bmag bn
Exceptions
|
|
ValueError, 'No data found at url' + str( url )
ValueError, 'error raised using url ' + str( url ) + ' ' + str( page )
ValueError, 'len(oneDParmList) != len(oneDParmValues)'
ValueError, 'unable to open url ' + str( url )
|
|
|
madCalculator2
|
madCalculator2 (
self,
year,
month,
day,
hour,
min,
sec,
latList,
lonList,
altList,
parms,
oneDParmList=[],
oneDParmValues=[],
twoDParmList=[],
twoDParmValues=[],
)
madCalculator2 is similar to madCalculator, except that a random collection of points in space can be specified,
rather than a grid of points. Also, a user can input 2D data.
Added to Madrigal2.6 as web service - will not run on earlier Madrigal installations.
Input arguments:
year - int
month - int
day - int
hour - int
min - int
sec - int
latList - a list of geodetic latitudes, -90 to 90
lonList - a list of longitudes, -180 to 180. Length must = lats
altList - a list of geodetic altitudes in km. Length must = lats
parms - comma delimited string of Madrigal parameters desired
oneDParmList - a list of one-D parameters whose values should
be set for the calculation. Can be codes or mnemonics.
Defaults to empty list.
oneDParmValues - a list of values (doubles) associated with the one-D
parameters specified in oneDParmList. Defaults to empty list.
twoDParmList - a python list of two-D parameters as mnemonics. Defaults to [].
twoDParmValues - a python list of lists of len = len(twoDParmList). Each individual
list is a list of doubles representing values of the two-D
parameter set in twoDParmList, with a length = number
of points (or equal to len(lats)). Defaults to [].
Returns:
A list of lists of doubles, where each list contains 3 + number of parameters doubles.
The first three doubles are the input latitude, longitude, and altitude. The rest of the
doubles are the values of each of the calculated values. If the value cannot be calculated,
it will be set to nan.
Example:
result = testData.madCalculator2(1999,2,15,12,30,0,[45,55],[-170,-150],[200,300],'sdwht,kp')
result = [ [1999.0, 2.0, 15.0, 12.0, 30.0, 0.0, 3.0, 15.0]
[1999.0, 2.0, 15.0, 12.0, 45.0, 0.0, 3.0, 15.0]
[1999.0, 2.0, 15.0, 13.0, 0.0, 0.0, 3.0, 15.0]
[1999.0, 2.0, 15.0, 13.0, 15.0, 0.0, 3.0, 15.0] ]
Columns: gdlat glon gdalt sdwht kp
Now uses POST to avoid long url issue
Exceptions
|
|
IOError, 'madCalculator2 requires Madrigal 2.6 or greater, but this site is version %s' %( self._madVers )
ValueError, 'No data found at url' + str( url )
ValueError, 'error raised using url ' + str( url ) + ' ' + str( page )
ValueError, 'len(oneDParmList) != len(oneDParmValues)'
ValueError, 'length of latList must be at least one'
ValueError, 'lengths of latList, lonList, altList must all be equal'
ValueError, 'unable to open url ' + str( url )
|
|
|
madCalculator3
|
madCalculator3 (
self,
yearList,
monthList,
dayList,
hourList,
minList,
secList,
latList,
lonList,
altList,
parms,
oneDParmList=[],
oneDParmValues=[],
twoDParmList=[],
twoDParmValues=[],
)
madCalculator3 is similar to madCalculator, except that multiple times can be specified,
where each time can have its own unique spatial positions and 1D and 2D parms. It is
equivalent to multiple calls to madCalculator2, except that it should greatly improve
performance where multiple calls to madCalculator2 are required for different times.
The only restriction is that the same parameters must be requested for every time.
Added to Madrigal2.6 as web service - will not run on earlier Madrigal installations.
Now uses POST to send arguments, due to large volume of data possible
Input arguments:
yearList - a list of years, one for each time requested (ints)
monthList - a list of months, one for each time requested. (ints)
dayList - a list of days, one for each time requested. (ints)
hourList - a list of hours, one for each time requested. (ints)
minList - a list of minutes, one for each time requested. (ints)
secList - a list of seconds, one for each time requested. (ints)
latList - a list of lists of geodetic latitudes, -90 to 90. The
first list is for the first time, the second for the second time, etc.
The list do not need to have the same number of points. The number
of times must match yearList.
Data organization: latList[timeIndex][positionIndex]
lonList - a list of lists of longitudes, -180 to 180. The
first list is for the first time, the second for the second time, etc.
The list do not need to have the same number of points. The number
of times must match yearList. Lens must match latList
Data organization: lonList[timeIndex][positionIndex]
altList - a list of lists of geodetic altitudes in km. The
first list is for the first time, the second for the second time, etc.
The list do not need to have the same number of points. The number
of times must match yearList. Lens must match latList
Data organization: altList[timeIndex][positionIndex]
parms - comma delimited string of Madrigal parameters desired
oneDParmList - a list of one-D parameters whose values should
be set for the calculation. Can be codes or mnemonics.
Defaults to empty list.
oneDParmValues - a list of lists of values (doubles) associated with the one-D
parameters specified in oneDParmList. Defaults to empty list.
The first list is for the first 1D parameter in oneDParmList,
and must be on length len(yearList). The second list is for
the second parameter, etc.
Data organization: onDParmValues[parameterIndex][timeIndex]
twoDParmList - a python list of of two-D parameters as mnemonics. Defaults to [].
twoDParmValues - a list of lists of lists of values (doubles) associated with the two-D
parameters specified in twoDParmList. Defaults to empty list.
The first list is for the first 2D parameter in oneDParmList,
and must be a list of length len(yearList). Each list in that
list must be of len(num positions for that time). The second list is for
the second parameter, etc.
Data organization: twoDParmValues[parameterIndex][timeIndex][positionIndex]
Returns:
A list of lists of doubles, where each list contains 9 + number of parameters doubles.
The first nine doubles are:
1) year, 2) month, 3) day, 4) hour, 5) minute, 6) second,
7) input latitude, 8) longitude, and 9) altitude.
The rest of the doubles are the values of each of the calculated values. If the value
cannot be calculated, it will be set to nan.
Example:
testData.madCalculator3(yearList=[2001,2001], monthList=[3,3], dayList=[19,20],
hourList=[12,12], minList=[30,40], secList=[20,0],
latList=[[45,46,47,48.5],[46,47,48.2,49,50]],
lonList=[[-70,-71,-72,-73],[-70,-71,-72,-73,-74]],
altList=[[145,200,250,300.5],[200,250,300,350,400]],
parms='bmag,pdcon,ne_model',
oneDParmList=['kinst','elm'],
oneDParmValues=[[31.0,31.0],[45.0,50.0]],
twoDParmList=['ti','te','ne'],
twoDParmValues=[[[1000,1000,1000,1000],[1000,1000,1000,1000,1000]],
[[1100,1200,1300,1400],[1500,1000,1100,1200,1300]],
[[1.0e10,1.0e10,1.0e10,1.0e10],[1.0e10,1.0e10,1.0e10,1.0e10,1.0e10]]])
Columns: year month day hour minute second gdlat glon gdalt bmag pdcon ne_model
Exceptions
|
|
IOError, 'madCalculator3 requires Madrigal 2.6 or greater, but this site is version %s' %( self._madVers )
ValueError, 'No data found at url' + str( url )
ValueError, 'Not all time lists have same length'
ValueError, 'error raised using url ' + str( url ) + ' ' + str( page )
ValueError, 'len(oneDParmList) != len(oneDParmValues)'
ValueError, 'mismatched number of points in altList'
ValueError, 'mismatched number of points in latList'
ValueError, 'mismatched number of points in lonList'
ValueError, 'unable to open url ' + str( url )
ValueError, 'wrong number of 1D parms for %s' %(str( parm ) )
|
ValueError, 'wrong number of 2D parms for %s' %(str( parm ) )
ValueError, 'yearList mus be a list, not %s' %(str( yearList ) )
|
|
|
|
madTimeCalculator
|
madTimeCalculator (
self,
startyear,
startmonth,
startday,
starthour,
startmin,
startsec,
endyear,
endmonth,
endday,
endhour,
endmin,
endsec,
stephours,
parms,
)
Input arguments:
startyear - int
startmonth - int
startday - int
starthour - int
startmin - int
startsec - int
endyear - int
endmonth - int
endday - int
endhour - int
endmin - int
endsec - int
stephours - float - number of hours per time step
parms - comma delimited string of Madrigal parameters desired (must not depend on location)
Returns:
A list of lists, where each list contains 6 ints (year, month, day, hour, min, sec) + number
of parameters. If the value cannot be calculated, it will be set to nan.
Example:
result = testData.madTestCalculator(1999,2,15,12,30,0,
1999,2,20,12,30,0,
24.0, kp,dst )
result = [[1999.0, 2.0, 15.0, 12.0, 30.0, 0.0, 3.0, -9.0]
[1999.0, 2.0, 16.0, 12.0, 30.0, 0.0, 1.0, -6.0]
[1999.0, 2.0, 17.0, 12.0, 30.0, 0.0, 4.0, -31.0]
[1999.0, 2.0, 18.0, 12.0, 30.0, 0.0, 6.7000000000000002, -93.0]
[1999.0, 2.0, 19.0, 12.0, 30.0, 0.0, 5.2999999999999998, -75.0]]
Columns: year, month, day, hour, min, sec, kp, dst
Exceptions
|
|
ValueError, 'No data found at url' + str( url )
ValueError, 'error raised using url ' + str( url ) + ' ' + str( page )
ValueError, 'unable to open url ' + str( url )
|
|
|
radarToGeodetic
|
radarToGeodetic (
self,
slatgd,
slon,
saltgd,
az,
el,
radarRange,
)
radarToGeodetic converts arrays of az, el, and ranges to geodetic locations.
Input arguments:
slatgd - radar geodetic latitude
slon - radar longitude
saltgd - radar altitude
az - either a single azimuth, or a list of azimuths
el - either a single elevation, or a list of elevations. If so, len(el)
must = len(az)
radarRange - either a single range, or a list of ranges. If so, len(radarRange)
must = len(az)
Returns:
A list of lists, where each list contains 3 floats (gdlat, glon, and gdalt)
Exceptions
|
|
ValueError, 'No data found at url' + str( url )
ValueError, 'all lists most have same length'
ValueError, 'error raised using url ' + str( url ) + ' ' + str( page )
ValueError, 'unable to open url ' + str( url )
|
|
|
simplePrint
|
simplePrint (
self,
filename,
user_fullname,
user_email,
user_affiliation,
)
simplePrint prints the data in the given file is a simple ascii format.
simplePrint prints only the parameters in the file, without filters or derived
parameters. To choose which parameters to print, to print derived parameters, or
to filter the data, use isprint instead.
Inputs:
filename - The absolute filename to be printed. Returned by getExperimentFiles.
user_fullname - full name of user making request
user_email - email address of user making request
user_affiliation - affiliation of user making request
Returns: string representing all data in the file in ascii, space-delimited form.
The first line if the list of parameters printed. The first six parameters will
always be year, month, day, hour, min, sec, representing the middle time of
the measurment.
|
|
traceMagneticField
|
traceMagneticField (
self,
year,
month,
day,
hour,
minute,
second,
inputType,
outputType,
alts,
lats,
lons,
model,
qualifier,
stopAlt=None,
)
traceMagneticField returns a point along a magnetic field line for each
point specified by the lists alts, lats, lons.
Traces to either 1) conjugate point, 2) intersection with a given altitude in the
northern or southern hemisphere, 3) to the apex, or 4) to GSM XY plane, depending on qualifier
argument. Uses Tsyganenko or IGRF fields, depending on model argument.
Input arguments are either GSM or Geodetic, depending on inputType argument.
Output arguments are either GSM or Geodetic, depending on outputType
argument.
Inputs:
year, month, day, hour, minute, second - time at which to do the trace
inputType - 0 for geodetic, 1 for GSM
outputType - 0 for geodetic, 1 for GSM
The following parameter depend on inputType:
alts - a list of geodetic altitudes or ZGSMs of starting point
lats - a clist of geodetic latitudes or XGSMs of starting point
lons - a list of longitude or YGSM of starting point
Length of all three lists must be the same
model - 0 for Tsyganenko, 1 for IGRF
qualifier - 0 for conjugate, 1 for north_alt, 2 for south_alt, 3 for apex, 4 for GSM XY plane
stopAlt - altitude in km to stop trace at, if qualifier is north_alt or south_alt.
If other qualifier, this parameter is not required. Default is None, which will raise
exception if qualifier is north_alt or south_alt
Returns a tuple of tuples, one tuple for point in (alts, lats, lons) lists, where each tuple has
three items:
geodetic altitude or ZGSM of ending point
geodetic latitude or XGSM of ending point
longitude or YGSM of ending point
If error, traceback includes error description
Exceptions
|
|
ValueError, 'No data found at url' + str( url )
ValueError, 'error raised using url ' + str( url ) + ' ' + str( page )
ValueError, 'stopAlt must be set for qualifer in (1,2)'
ValueError, 'unable to open url ' + str( url )
|
|
|