HMI_Logo HMI Ring Diagrams
Pipeline Module Specifications

datavg

Module Version
	1.0	14 XI 2011

General Description
	datavg takes as its input a dataset of records containing at
	least one data segment of structured data (FITS NAXIS > 0) and the name
	of a data series to which it writes a record containing at least one
	and up to three segments of the same structure as the input records and
	containing the per-pixel mean, variance, and count of valid values,
	respectively.

Status
	This version has been superseded by Version 1.1

	Included in JSOC release 6.0
	In use for ring-diagram pipelines from 2011.06.24 through 2012.08.03
	This version supersedes Version 0.9

	The source code and Makefile are in the JSOC CVS repository

Usage
	datavg [-nov] in= DataDescriptor out= DataSeries
	    [param= val ...]
	
Flags
	-n	no output records produced, diagnostics only
	-o	remove effects of observer velocity
	-v	run in verbose mode

Parameters
	in		Either the name of an input data set (record selection)
			or the name of an input data series; in the latter case,
			tmid and length parameters must also be supplied.
			No default, must be provided.

	out		The name of the output data series. No default, must be
			supplied even if run with the -n flag.

	tmid		The target time at the midpoint of the averaging interval,
			in Carrington CR:CL form; Default: Not Specified.

	length		The length of the averaging interval, in degrees of
			Carrington rotation. Default: Not Specified

	count		The name of the segment in the output series expected to
			contain the per-pixel valid data counts. Default:  valid

	mean		The name of the segment in the output series expected to
			contain the per-pixel means. Default: mean

	power		The name of the segment in the output series expected to
			contain the per-pixel variances. Default: power

	reject		The name of a file containing a list of records in the
			input data series to be rejected regardless of their
			quality mask values. The format of this file is to be specified.
			Default: Not Specified, implying no rejection list

	qmask		A 32-bit quantity specifying acceptable values of the quality
			value of the key associated with qual_key. If the bitwise
			and of qmask and the quality value for an input record is
			non-zero, that record will be skipped (treated as if it
			contained no data).Default: 0x80000000

	copy		A comma separated list of keywords, optionally terminated with
			',+', whose values are to be propagated to the output record
			as-is from the first input record in the input data set.
			Default: + implying the following default list:
			"TELESCOP", "INSTRUME", "WCSNAME", "WCSAXES"

	average		A comma separated list of keywords, optionally terminated with
			',+', whose values are to be averaged from the accepted input
			records for propagation to the output record. If the keyword
			'+' occurs in the list, the following default set of keywords
			will be averaged in addition to any others listed: SAT_ROT,
			OBS_VR, OBS_VW, OBS_VN. N.B.: The WCS keywords CRVALn, CRPIXn,
			CDELTn, and CROTAn, along with the tobs_key are always averaged.
			For the averaged keyword XXX an additional RMS keyword
			D_XXX will also be set. Default: "+"

	mscale		The FITS BSCALE value to be applied to the output mean segment.
			Default: Segment Default, implying use of the defalut value
			for the relevant segment in the output series

	mzero		The FITS BZERO value to be applied to the output mean segment.
			Default: Segment Default, implying use of the defalut value
			for the relevant segment in the output series

	pscale		The FITS BSCALE value to be applied to the output power segment.
			Default: Segment Default, implying use of the defalut value
			for the relevant segment in the output series

	pzero		The FITS BZERO value to be applied to the output power segment.
			Default: Segment Default, implying use of the defalut value
			for the relevant segment in the output series

	pkey		The prime key name for the index over which records are selected
			for averaging. Default: T_OBS

	qual_key	The key name for a 32-bit image quality field in the input data
			series;  Default: Quality

	pa		The centre of acceptable roll angle values of the roll_key (deg);
			Default: 180.0

	dpa		The maximum deviation of acceptable roll angle values of the
			roll_key (deg) from pa; records with roll-angles outside this
			range will be skipped. Default: 1.0

Input Data series class:
	In principle, any series containing at least one data segment of structured data
	(a hypercube, FITS NAXIS > 0); if the series contains multiple segments (of any
	type!), the segment to be averaged must be specified as part of the input dataset
	name.

Sample input series:
	hmi.V_45s, hmi.rdVpspec_fd15, hmi.rdVpspec_fd30

Input keys used or inspected:
	OBS_VR, T_REC_index, WCSAXES

Output Data series class:
	Data statistics hypercubes: expected to contain up to three segments,
	named as per the count, mean, and power parameters, with same rank and
	size (if not vardim) as the input series.

Sample output series:
	hmi.V_avg120, hmi.rdVavgpspec_fd15, hmi.rdVavgpspec_fd30

Output keys set (if possible):
	CarrRot, CMLon, MidTime, Module, BLD_VERS, Source, Input, Created, Interval,
	CTYPEn, CUNITn, COMMENT, HISTORY, DataRecs, MissRecs, T_FIRST, T_LAST,
	CRPIXn, D_CRPIXn, CRVALn, D_CRVALn, CDELTn, D_CDELTn, CROTAn, D_CROTAn,
	CRLN_OBS, D_CRLN_OBS, LOG_BASE, CountMIN, CountMAX, MeanMIN, MeanMAX, MeanAVG,
	MeanRMS, MeanSKEW, MeanKURT, PowrMIN, PowrMAX, PowrAVG, PowrRMS, PowrSKEW,
	PowrKURT (plus any keys [and D_keys as appropriate] as specified by copy
	and average parameter values)

Sample commands:
	datavg in= hmi.V_45s out= hmi.V_avg120 tmid= 2116:360 length= 120 \
	    qmask= 0x80004000 reject= ~jsoc/hmi/tables/LHS_reject.v \
	    mscale= 0.25 mzero= 0.0 pscale= 10.0 pzero= 32767.0 \
	    average= "T_OBS,DSUN_OBS,CRLT_OBS,RSUN_OBS,OBS_VR,OBS_VW,OBS_VN" 

	datavg out= hmi.rdVavgpspec_fd15 mean= logP pkey= CMLon \
	    mscale= 0.0005 mzero= -11.5 \
	    in= hmi.rdVpspec_fd15[2097][][][+45.0][-30.0] \
	    copy= "CarrRot,LatHG,LonCM,DELTA_K,DELTA_NU,D_OMEGA,Apode_f,APOD_MIN,APOD_MAX" 

Bugs in current version:
	The tmid target for the middle of the averaging interval can only be expressed
	as a Carrington longitude of central meridian, assumed geocentric, and the
	length parameter can only be specified in units of Carrington degrees

	No account is taken of periodicity in the averaging of angular key values;
	for example, a set of values that cluster symmetrically around 180 may
	average to 0.

	The averaging of the roll angle does not work properly when the angles wrap
	round 360

	The check on nominal roll angle assumes that the keyword is CROTA2, and that
	the units are degrees

	The application of the optional orbital velocity correction is purely scalar;
	only the value of OBS_VR is subtracted from all image values, regardless of
	image location, and transverse components of the orbital velocity are ignored.

	No check is performed to verify that values of keywords copied as-is from
	the first record in the dataset do not vary.

	The presence of generic segments in the input data series should not require
	specification of the segment name in the input data set.

	The reported extremal values of mean and power are determined before possible
	output scaling; they may be outside the representable range

	If the input dataset is specified explicitly in the in string, the keywords
	for CarrRot and CMLon are set to garbage, and there is no checking that the
	length is correct.

	If the input record set is not explicitly specified, the records must be
	uniquely tagged by either time or Carrington rotation and longitude.
	In the former case, geocentric physical ephemerides for the Sun are assumed.

Significant changes from Previous Version
	— changed argument tobs_key to pkey; generalized treatment of prime key to
	different types

	— added keywords for output segment names to override the default values,
	and optional scaling factors and offset values for the mean and power segments;
	allow for output series without all three segments

	— added setting of key values for T_FIRST, T_LAST, averaging of CRLN_OBS

Valid HTML 4.01 Transitional HMI Ring Diagrams 26 Mar 2018, 17:31-0700