|
HMI Ring Diagrams
Pipeline Module Specifications
|
datavg
Module Version
1.6 2 X 2017
General Description
datavg takes as its input a dataset of records containing at
least one data segment of structured data (FITS NAXIS > 0) and the name
of a data series to which it writes a record containing at least one
and up to three segments of the same structure as the input records and
containing the per-pixel mean, variance, and count of valid values,
respectively.
Status
Not scheduled for inclusion in any JSOC release
In use for ring-diagram pipelines on 2017.10.02 only
for hmi.V_avg120[2195][240])
This version involved a special call for pre-staging records suggested
for improved efficiency in the database, but the efficiency actually
declined, so the version was withdrawn.
The source code and Makefile is in ~rick/src/drms/versions/datavg/
This version has been superseded by Version 1.7
Usage
datavg [-nov] in= DataDescriptor out= DataSeries
[param= val ...]
Flags
-n no output records produced, diagnostics only
-o remove effects of observer velocity
-v run in verbose mode
Parameters
in Either the name of an input data set (record selection)
or the name of an input data series; in the latter case,
tmid and length parameters must also be supplied.
No default, must be provided.
out The name of the output data series. No default, must be
supplied even if run with the -n flag.
tmid The target time at the midpoint of the averaging interval,
in either DATE_TIME or Carrington CR:CL form.
Default: Not Specified.
length The length of the averaging interval, in degrees of
Carrington rotation or seconds, depending on the format
of tmid. Default: Not Specified
count The name of the segment in the output series expected to
contain the per-pixel valid data counts. Default: valid
mean The name of the segment in the output series expected to
contain the per-pixel means. Default: mean
power The name of the segment in the output series expected to
contain the per-pixel variances. Default: power
log The name of the generic segment in the output series expected
contain the run log. Default: Log
reject The name of a file containing a list of records in the
input data series to be rejected regardless of their
quality mask values. The format of this file is to be specified.
Default: Not Specified, implying no rejection list
qmask A 32-bit quantity specifying acceptable values of the quality
value of the key associated with qual_key. If the bitwise
and of qmask and the quality value for an input record is
non-zero, that record will be skipped (treated as if it
contained no data).Default: 0x80000000
copy A comma separated list of keywords, optionally terminated with
',+', whose values are to be propagated to the output record
as-is from the first input record in the input data set.
Default: + implying the following default list:
"TELESCOP", "INSTRUME", "WCSNAME", "WCSAXES"
average A comma separated list of keywords, optionally terminated with
',+', whose values are to be averaged from the accepted input
records for propagation to the output record. If the keyword
'+' occurs in the list, the following default set of keywords
will be averaged in addition to any others listed: OBS_VR,
OBS_VW, OBS_VN. N.B.: The WCS keywords CRVALn, CRPIXn, CDELTn,
and CROTAn, along with the tobs_key are always averaged.
For the averaged keyword XXX an additional RMS keyword
D_XXX will also be set. Default: "+"
mscale The FITS BSCALE value to be applied to the output mean segment.
Default: Segment Default, implying use of the defalut value
for the relevant segment in the output series
mzero The FITS BZERO value to be applied to the output mean segment.
Default: Segment Default, implying use of the defalut value
for the relevant segment in the output series
pscale The FITS BSCALE value to be applied to the output power segment.
Default: Segment Default, implying use of the defalut value
for the relevant segment in the output series
pzero The FITS BZERO value to be applied to the output power segment.
Default: Segment Default, implying use of the defalut value
for the relevant segment in the output series
pkey The prime key name for the index over which records are selected
for averaging. Default: T_OBS
qual_key The key name for a 32-bit image quality field in the input data
series; Default: Quality
roll_key The key name for the input roll-angle field; Default: CROTA2
pa The centre of acceptable roll angle values of the roll_key (deg);
Default: 180.0
dpa The maximum deviation of acceptable roll angle values of the
roll_key (deg) from pa; records with roll-angles outside this
range will be skipped. Default: 1.0
cvkey The key name to be used for the key value representing the
calibration version in the input data series to be tested
against values of cvok and cvno; Default: CalVer64
cvok The acceptable value of cvkey; Default: any
cvno The unacceptable value of cvkey; Default: none
setkey The name of a special extra key to be set to the value setval
in the output record; Default: Not Specified
setval The value to which the keyword setkey is to be set in the
output record; Default: Not Specified
Input Data series class:
In principle, any series containing at least one data segment of structured data
(a hypercube, FITS NAXIS > 0); if the series contains multiple segments (of any
type!), the segment to be averaged must be specified as part of the input dataset
name.
Sample input series:
hmi.V_45s, hmi.rdVpspec_fd05, hmi.rdVpspec_fd15, hmi.rdVpspec_fd30
Input keys potentially used or inspected:
CAR_ROT, CDELT*, CRLN_OBS, CROTA*, CRPIX*, CRVAL*, CTYPE*, CUNIT*,
LOG_BASE, OBS_VR, T_REC_index, WCSAXES (plus those specified as
calling parameters)
Output Data series class:
Data statistics hypercubes: expected to contain up to three segments,
named as per the count, mean, and power parameters, with same rank and
size (if not vardim) as the input series, and an optional generic segment
named as per the log parameter.
Sample output series:
hmi.V_avg120, hmi.rdVavgpspec_fd05, hmi.rdVavgpspec_fd15, hmi.rdVavgpspec_fd30
Output keys set (if possible):
CarrRot, CMLon, MidTime, Module, BLD_VERS, Source, Input, Created, Interval,
CTYPEn, CUNITn, COMMENT, HISTORY, DataRecs, MissRecs, T_FIRST, T_LAST,
CRPIXn, D_CRPIXn, CRVALn, D_CRVALn, CDELTn, D_CDELTn, CROTAn, D_CROTAn,
CRLN_OBS, D_CRLN_OBS, LOG_BASE, CountMIN, CountMAX, MeanMIN, MeanMAX, MeanAVG,
MeanRMS, MeanSKEW, MeanKURT, PowrMIN, PowrMAX, PowrAVG, PowrRMS, PowrSKEW,
PowrKURT (plus any keys [and D_keys as appropriate] as specified by copy,
average, and setkey parameter values)
Sample commands:
datavg in= hmi.V_45s out= hmi.V_avg120 tmid= 2161:300 length= 120 \
qmask= 0x80004000 reject= ~jsoc/hmi/tables/LHS_reject.v \
mscale= 0.25 mzero= 0.0 pscale= 10.0 pzero= 32767.0 \
average= "T_OBS,DSUN_OBS,CRLT_OBS,RSUN_OBS,OBS_VR,OBS_VW,OBS_VN"
datavg out= hmi.rdVavgpspec_fd15 mean= logP pkey= CMLon \
mscale= 0.0005 mzero= -11.5 \
in= hmi.rdVpspec_fd15[2160][][][+45.0][-30.0] \
copy= "CarrRot,LatHG,LonCM,DELTA_K,DELTA_NU,D_OMEGA,Apode_f,APOD_MIN,APOD_MAX"
Bugs in current version:
The application of the optional orbital velocity correction is purely scalar;
only the value of OBS_VR is subtracted from all image values, regardless of
image location, and transverse components of the orbital velocity are ignored.
No check is performed to verify that values of keywords copied as-is from
the first record in the dataset do not vary.
The presence of generic segments in the input data series should not require
specification of the segment name in the input data set.
The reported extremal values of mean and power are determined before possible
output scaling; they may be outside the representable range
If the input dataset is specified explicitly in the in string, the keywords
for CarrRot and CMLon are set to garbage, and there is no checking that the
length is correct.
If the input record set is not explicitly specified, the records must be
uniquely tagged by either time or Carrington rotation and longitude.
In the former case, geocentric physical ephemerides for the Sun are assumed.
Significant changes from Previous Version
None; added stage_records call to pre-cache SUMS info