HMI_Logo HMI Ring Diagrams
Pipeline Module Specifications

rdcover

Module Version
	1.0	21 November 2013

General Description
	rdcover generates estimates of the effective duty cycle (coverage)
	in its input data source for a time range, cadence, and selection
	criteria identical to those that would be supplied to rdcover.
	It is intended for use in filtering front ends for pipeline scripts
	that automatically process data over extended time ranges.

	The module output is in the form of lines of text sent to stdout
	and expected to be parsed by a script.

	The module is effectively a subset of mtrack, and might be better
	implemented as a flag option to that module. Except for the difference in
	name of the ds parameter, all parameters should have the same meaning
	as their equivalents in mtrack

Status
	Included in JSOC release 8.3
	In use for ring-diagram pipelines from about 2013.06.10
	This version supersedes Version 0.9

	The source code and Makefile are in the JSOC CVS repository


Usage
	rdcover [-flags] ds= TargetTime
	    [param= value ...]
	
Flags
	-v	run verbose

Parameters
	ds		either the name of an input data set (record selection)
			or the name of an input data series; in the latter case,
			either tmid and length parameters or tstart and tstop
			must also be supplied. No default, must be provided.

	reject		The name of a file containing a list of records in the
			input data series to be rejected for various reasons.
			This is a list that can be prepared supplementary to
			whatever quality/rejection criteria may be contained in
			the records. Default: Not Specified.
			The format of this file is to be specified.

	qmask		A 32-bit quantity specifying acceptable values of the quality
			value of the key associated with qual_key. If the bitwise
			and of qmask and the quality value for an input record is
			non-zero, that record will be skipped (treated as if it
			contained no data). Default: 0x8000000

	max_miss	The tolerance threshold for the value of keyword MISSVALS.
			If the value for an input record exceeds this value, that record
			will be skipped (treated as if it contained no data).Default: 0

	tmid		The target midpoint of the tracking interval, which may
			be expressed in either standard Date_Time format or in
			CarringtonRotation:Longitude format, e.g. 1988:150.0
			in which case it refers to the central meridian crossing
			time appropriate to the observing platform.
			Must be used in conjunction with the length parameter.
			Ignored if the in value is a specific data recordset.

	length		The length of the tracking interval, in units of the tstep
			parameter, hence it corresponds to the number of timesteps
			in the output data cube. Must be used in conjunction with the
			tmid parameter. Ignored if the in value is a specific data
			recordset.

	tstart		The target starting time of the tracking interval, expressed
			in standard Date_Time format. Must be used in conjunction with
			the tstop parameter. Ignored if the in value is a specific
			data recordset, or if the tmid parameter is specified..

	tstop		The target ending time of the tracking interval, expressed
			in standard Date_Time format. Must be used in conjunction with
			the tstart parameter. Ignored if the in value is a specific
			data recordset, or if the tmid parameter is specified..

	tstep		The temporal cadence of the output tracked cubes, in sec.
			If unspecified, defaults to the cadence of the input series.

	trec_key	The key name to be used for the prime key of the input data
			series, assumed to be a slotted time series; Default: T_REC

	tobs_key	The key name to be used for the key value representing the
			midpoint of the record observation times in the input data
			series; Default: T_OBS

	tstp_key	The key name to be used for the key value representing the
			trec_key step size in the input data series; Default: CADENCE

	qual_key	The key name for a 32-bit image quality field in the input data
			series;  Default: Quality.not currently implemented

	clon_key	The key name to be used for the key value representing the
			observer Carrington longitude at the time of observation in
			the input data series; Default: CRLN_OBS

	clat_key	The key name to be used for the key value representing the
			observer heliographic latitude at the time of observation in
			the input data series; Default: CRLT_OBS

	crot_key	The key name to be used for the key value representing the
			observer Carrington rotation number at the time of observation
			in the input data series; Default: CAR_ROT

	pa		The central value of acceptable roll angles (position angle of
			solar north) in degrees. N.B. As with mtrack, the position angle
			is interpreted as being measured positive westward from north
			(clockwise), i.e. contrary to the AIPS convention.
			Default: Not Specified (in which case all values are accepted)

	dpa		The range of acceptable roll angles, in degrees;
			Default: Not Specified (in which case all values are accepted)

	pa_key		The key name to be used for the key value representing the
			position angle of solar north on the image in the input data series;
			Default: CROTA2

Sample output:
1=> rdcover ds= hmi.V_45s tmid= 2010.07.08_00:00:00_TAI length= 1920 reject= ~rick/hmi/qual/reject.V max_miss= 0
1818 of 1920 possible input records accepted
    effective coverage = 0.947
    9 input records rejected for quality matching 80000000
    93 input records rejected for missing values exceeding 0

2=> rdcover ds= hmi.V_45s"[2010.07.07_12:00:00_TAI/1d]" reject= ~rick/hmi/qual/reject.V max_miss= 0
1819 of 1920 possible input records accepted
    effective coverage = 0.947
    9 input records rejected for quality matching 80000000
    92 input records rejected for missing values exceeding 0

3=> rdcover ds= hmi.V_45s"[2010.07.07_12:00:45_TAI/1d]" reject= ~rick/hmi/qual/reject.V max_miss= 0
1818 of 1920 possible input records accepted
    effective coverage = 0.947
    9 input records rejected for quality matching 80000000
    93 input records rejected for missing values exceeding 0

4=> rdcover ds= hmi.V_45s tmid= 2010.07.08_00:00 length= 1920 qmask= 0x00010000
1788 of 1920 possible input records accepted
    effective coverage = 0.931
    9 input records rejected for invalid values of T_OBS
    37 input records rejected for quality matching 00010000
    86 input records rejected for missing values exceeding 0
5=> rdcover ds= hmi.V_45s tmid= 2010.07.25_TAI length= 1920 reject= ~rick/hmi/qual/reject.V max_miss= 5
1892 of 1920 possible input records accepted
    effective coverage = 0.985
    19 input records rejected for quality matching 80000000
    7 input records rejected for missing values exceeding 5
    2 input records rejected from rejection list

6=> rdcover -v ds= hmi.V_45s tmid= 2010.07.25_TAI length= 1920 reject= ~rick/hmi/qual/reject.V max_miss= 5 tobs_key= T_REC
rdcoverage v 0.8: JSOC version V5R14
checking data from 2010.07.24_11:59 - 2010.07.25_11:58 at cadence of 45.0 s
         1920 data records
2010.07.24_13:37:30_TAI: MISSVALS = 6 > 5
step 130 to be interpolated from images 2010.07.24_13:36:11 and 2010.07.24_13:37:41
2010.07.24_16:22:30_TAI: MISSVALS = 6 > 5
step 350 to be interpolated from images 2010.07.24_16:21:11 and 2010.07.24_16:22:41
2010.07.24_16:37:30_TAI: MISSVALS = 7 > 5
step 370 to be interpolated from images 2010.07.24_16:36:11 and 2010.07.24_16:37:41
2010.07.24_17:32:15_TAI: Quality = c0000000 matches 80000000
2010.07.24_17:33:00_TAI: Quality = c0000000 matches 80000000
2010.07.24_17:33:45_TAI: Quality = c0000000 matches 80000000
2010.07.24_17:34:30_TAI: Quality = c0000000 matches 80000000
2010.07.24_17:35:15_TAI: Quality = c0000000 matches 80000000
2010.07.24_17:36:00_TAI: Quality = c0000000 matches 80000000
2010.07.24_17:36:45_TAI: Quality = c0000000 matches 80000000
2010.07.24_17:37:30_TAI: Quality = c0000000 matches 80000000
2010.07.24_17:38:15_TAI: Quality = c0000000 matches 80000000
2010.07.24_17:39:00_TAI: Quality = c0000000 matches 80000000
2010.07.24_17:39:45_TAI: Quality = c0000000 matches 80000000
2010.07.24_17:40:30_TAI: Quality = c0000000 matches 80000000
2010.07.24_17:41:15_TAI: Quality = c0000000 matches 80000000
2010.07.24_17:42:00_TAI: Quality = c0000000 matches 80000000
2010.07.24_17:42:45_TAI: Quality = c0000000 matches 80000000
2010.07.24_17:43:30_TAI: Quality = c0000000 matches 80000000
2010.07.24_17:44:15_TAI: Quality = c0000000 matches 80000000
2010.07.24_17:45:00_TAI: Quality = c0000000 matches 80000000
2010.07.24_17:45:45_TAI: Quality = c0000000 matches 80000000
step 442 to be interpolated from images 2010.07.24_17:30:11 and 2010.07.24_17:46:41
step 443 to be interpolated from images 2010.07.24_17:30:11 and 2010.07.24_17:46:41
step 444 to be interpolated from images 2010.07.24_17:30:11 and 2010.07.24_17:46:41
step 445 to be interpolated from images 2010.07.24_17:30:11 and 2010.07.24_17:46:41
step 446 to be interpolated from images 2010.07.24_17:30:11 and 2010.07.24_17:46:41
step 447 to be interpolated from images 2010.07.24_17:30:11 and 2010.07.24_17:46:41
step 448 to be interpolated from images 2010.07.24_17:30:11 and 2010.07.24_17:46:41
step 449 to be interpolated from images 2010.07.24_17:30:11 and 2010.07.24_17:46:41
step 450 to be interpolated from images 2010.07.24_17:30:11 and 2010.07.24_17:46:41
step 451 to be interpolated from images 2010.07.24_17:30:11 and 2010.07.24_17:46:41
step 452 to be interpolated from images 2010.07.24_17:30:11 and 2010.07.24_17:46:41
step 453 to be interpolated from images 2010.07.24_17:30:11 and 2010.07.24_17:46:41
step 454 to be interpolated from images 2010.07.24_17:30:11 and 2010.07.24_17:46:41
step 455 to be interpolated from images 2010.07.24_17:30:11 and 2010.07.24_17:46:41
step 456 to be interpolated from images 2010.07.24_17:30:11 and 2010.07.24_17:46:41
step 457 to be interpolated from images 2010.07.24_17:30:11 and 2010.07.24_17:46:41
step 458 to be interpolated from images 2010.07.24_17:30:11 and 2010.07.24_17:46:41
step 459 to be interpolated from images 2010.07.24_17:30:11 and 2010.07.24_17:46:41
step 460 to be interpolated from images 2010.07.24_17:30:11 and 2010.07.24_17:46:41
step 461 to be interpolated from images 2010.07.24_17:30:11 and 2010.07.24_17:46:41
step 462 to be interpolated from images 2010.07.24_17:30:11 and 2010.07.24_17:46:41
2010.07.25_00:39:00_TAI: MISSVALS = 7 > 5
step 1012 to be interpolated from images 2010.07.25_00:37:41 and 2010.07.25_00:39:11
2010.07.25_01:09:00_TAI: MISSVALS = 7 > 5
step 1052 to be interpolated from images 2010.07.25_01:07:41 and 2010.07.25_01:09:11
2010.07.25_10:40:30_TAI: MISSVALS = 6 > 5
step 1814 to be interpolated from images 2010.07.25_10:39:11 and 2010.07.25_10:40:41
2010.07.25_11:53:15_TAI: MISSVALS = 7 > 5
step 1911 to be interpolated from images 2010.07.25_11:51:56 and 2010.07.25_11:53:26
1892 of 1920 possible input records accepted
    effective coverage = 0.985
    19 input records rejected for quality matching 80000000
    7 input records rejected for missing values exceeding 5
    2 input records rejected from rejection list

7=> rdcover ds= hmi.V_45s tmid= 2010.07.08_00:00 length= 1920 qmask= 0x00010000
1788 of 1920 possible input records accepted
    effective coverage = 0.931
    9 input records rejected for invalid values of T_OBS
    37 input records rejected for quality matching 00010000
    86 input records rejected for missing values exceeding 0

8=> rdcover ds= hmi.V_45s"[2010.07.07_17:30_TAI/2h]" qmask= 0x00010000 max_miss= 100
120 of 160 possible input records accepted
    effective coverage = 0.750
    9 input records rejected for invalid values of T_OBS
    31 input records rejected for quality matching 00010000

9=> rdcover ds= hmi.V_45s"[2010.07.07_17:30_TAI/2h]" qmask= 0x00010000 max_miss= 100 tobs_key= T_REC
129 of 160 possible input records accepted
    effective coverage = 0.806
    31 input records rejected for quality matching 00010000

=> rdcover ds= mdi.fd_V_lev18 tmid= 1988:180.0 length= 1664 \
    reject= ~rick/soi/qual/reject.fd_V_01h
1573 of 1664 possible input records accepted
    effective coverage = 0.945
    73 input records rejected for invalid values of T_OBS
    17 input records rejected for missing values exceeding 0
    1 input records rejected from rejection list

=> rdcover -v ds= mdi.fd_V_lev18 tmid= 1988:180.0 length= 256 \
    reject= ~rick/soi/qual/reject.fd_V_01h
rdcoverage v 0.7: JSOC version V5R8X
Target time 1988:180.0 = 2002.04.12_16:19:27 adjusted to 2002.04.12_16:19:28
checking data from 2002.04.12_14:11 - 2002.04.12_18:26 at cadence of 60.0 s
2002.04.12_14:24:00_TAI: T_OBS invalid
2002.04.12_14:54:00_TAI: MISSVALS = 12592 > 0
2002.04.12_16:00:00_TAI: T_OBS invalid
2002.04.12_16:06:00_TAI: MISSVALS = 638999 > 0
2002.04.12_17:05:00_TAI: MISSVALS = 156341 > 0
2002.04.12_17:36:00_TAI: T_OBS invalid
249 of 256 possible input records accepted
    effective coverage = 0.973
    3 input records rejected for invalid values of T_OBS
    3 input records rejected for missing values exceeding 0
    1 input records rejected from rejection list

Bugs in current version
	If the data set is explicitly specified rather than restricted by the
	values of tmid and length, the reported coverage may not be exactly
	the same, due to interpolation to the target center (see examples 1–3).

	If the selected timestep does not match the data cadence of the input
	data series, the reported coverage is erroneous.

	If there are fewer than 2 records (acceptable or not) in the dataset record
	selection, only one line is printed; it would be better for purposes of
	output parsing to print at least the first two lines in standard format

No significant changes from Previous Version


Valid HTML 4.01 Transitional HMI Ring Diagrams 14 Jan 2015, 13:20-0800