HMI Ring Diagrams Pipeline Data Series Specifications

Data Series class

(tracked) (mapped) data cubes

• hmi.rdVtrack_fd05 [CarrRot] [CMLon] [LonHG] [LatHG] [LonCM]
• hmi.rdVtrack_fd15 [CarrRot] [CMLon] [LonHG] [LatHG] [LonCM]
• hmi.rdVtrack_fd30 [CarrRot] [CMLon] [LonHG] [LatHG] [LonCM]
• hmi.tdVtrack_synopHC [MidTime] [LonHG] [LatHG] [LonCM]

Individual Data Series notes

Records hmi.rdVtrack_*[2096-2125] and hmi.tdVtrack_synopHC[2010.04.30-2012.06.30] originally used values of CROTA2 in the input data that were subsequently modified by subtraction of 0°.07 (going from a nominal 180° to 179°.93). Thus there is expected to be a corresponding rotation of the north axis and a shift in the location of the mapped data in these records beginning at 2126:360 or 2012.07.01. All records in the series hmi.rdVtrack_*[2096-2125] have been subsequently reprocessed using the corrected values of CROTA2.

Records hmi.rdVtrack_*[2096-2234] and hmi.tdVtrack_synopHC[2010.04.30-2020.08.31] originally used values of CRLN_OBS in the input data that should have been modified by subtraction of the value 0°.081894. (Data from 2020.11.15_00:00_TAI forward in the relevant series are so corrrected.) Records hmi.rdVtrack_*[2096-2125] have been subsequently reprocessed using the corrected values of CRLN_OBS

Data Record organization

Individual records correspond to data cubes with a unique set of the union of the prime keyword values. In general, individual records should be referenced by three prime keys describing their spatial and temporal locations. Additional or diferent prime keys may be necessary if the data series mixes tracked data for the same time and location, but different other parameters, such as observable, data source, region size, length of tracking interval, tracking rate, etc.

Data Segments

The data segments, the actual tracked data cubes, are expected to be stored as either bin or fits protocol files with the data stored as 16-bit fixed point numbers with appropriate BSCALE and BZERO values. (Other storage classes are possible, but the mtrack module unconditionally calculates the output data as floats internally, thus limiting the precision.) The files can be stored with fits (compress) or bin.gz protocols, the former achieves compression of about 35% in typical realistic cases. For a discussion of the various segment protocols, see segments.

There is no restriction on the name of the output data segment. If more than one 3-dimensional segment is present in the series, mtrack will write to the first one that is either vardim or variable with axes matching the run parameters.

A data segment representing the 3-dimensional data cube output from mtrack is essential. mtrack also recognizes the possible existence of another generic protocol segment named Log. If present, and if mtrack is run with the -v option, run information (including whether individual planes are directly mapped, interpolated, or extrapolated) will be written to its file.

It would be possible to put additional data in other segments that would be unwieldy to store in the database. An example that comes to mind is the window function from the tracking, if that would be useful. This is not currently supported by mtrack however.

Prime Keywords

The following prime keys are expected to be common to data series in this class:

	MidTime, time,   "Midpoint of tracking interval"
	CarrRot, int,    "Carrington rotation at MidTime"
	CMLon,   float,  "Longitude of central meridian at MidTime"
	LonHG,   float,  "Carrington longitude of tracked region center"
	LatHG,   float,  "Latitude of tracked region center"
	LonCM,   float,  "CM (Stonyhurst) longitude of tracked region center at MidTime"

Not all of the first three keywords specifying the central time of the tracking interval need be prime keys, nor even present; in fact, they are clearly redundant. It suffices that either MidTime or both CarrRot and CMLon be present as prime keys. (Another possible prime keyword is CarrTime, but owing to the difficulty of searching on strings its use is discouraged, at least until the #() construct for time queries is extended to include this format.) Likewise, one of the three possible prime keys CMLon, LonHG, and LonCM is redundant, but it may be useful to keep all three for data selection. Default values and format specifications can of course be tailored to the specific data series. Likewise, the types of CMLon, LonHG, LonCM, and LatHG can be integer, and any of these (as well as MidTime) can be slotted.

Additional Keywords

The following keys may occurr in data series in this class, and if present will be appropriately set by mtrack from required or optional input parameters and data:

	CTYPE1,   string, "WCS axis 1 type"
	CTYPE2,   string, "WCS axis 2 type"
	CUNIT1,   string, "WCS axis 1 units"
	CUNIT2,   string, "WCS axis 2 units"
	CUNIT3,   string, "WCS axis 3 units"
	CRPIX1,   float,  "WCS reference pixel 1 location"
	CRPIX2,   float,  "WCS reference pixel 2 location"
	CRPIX3,   float,  "WCS reference pixel 3 location"
	CDELT1,   float,  "WCS coordinate 1 increment at reference point"
	CDELT2,   float,  "WCS coordinate 2 increment at reference point"
	CDELT3,   float,  "WCS coordinate 3 increment at reference point"
	CRVAL1,   float,  "WCS reference point 1 value"
	CRVAL2,   float,  "WCS reference point 2 value"
	CRVAL3,   double, "WCS reference point 3 value"
	CarrTime, string, "Carrington rotation and central longitude at MidTime"
	CarrLon,  double, "Carrington longitude at MidTime (deg of rotation since epoch)
	T_START,  time,   "Start of tracking interval"
	T_STOP,   time,   "End of tracking interval"
 	Duration, int,	  "Length of tracking interval"
	Cadence,  float,  "Temporal image cadence"
	LonSpan,  float,  "Central longitude span of tracking interval"
	MapProj,  string, "Map projection"
	MapScale, float,  "Map scale"
	Map_PA,   float,  "Position angle of Carrington north wrt axis 2"
	Size,     float,  "Diameter of region (nominal)"
	Width,    float,  "Equatorial longitude extent of region (nominal)"
	Height,   float,  "Latitude extent of region (nominal)"
	Coverage, float,  "Duty cycle (valid images/total frames)"
	Interp,   string, "Interpolation option"
	ZonalTrk, float,  "Zonal tracking rate (rel to Carrington)"
	ZonalVel, float,  "Zonal tracking velocity at RSunRef and LatHG" 
	LonDrift, float,  "Longitude drift over tracking interval"
	MeridTrk, float,  "Meridional tracking rate"
	MeridVel, float,  "Meridional tracking velocity at RSunRef"
	LatDrift, float,  "Latitude drift over tracking interval"
	RSunRef,  float,  "Reference value of solar radius"
	Interp,   string, "Interpolation scheme"
	Module,   string, "Creation module"
	Source,   string, "Input data source"
	Input,    string, "Input data records"
	Backgrnd, string, "Background image subtracted from data"
	Created,  time,   "Record creation time"
	BLD_VERS, string, "JSOC library build"
	RejectList, string, "Image rejection list"
	MDI_PA_Corr, float, "Correction to MDI Position Angle"
	MeanMU,   float,  "Mean center-to-limb distance"
	MeanLat,  float,  "Mean latitude of mapped region"
	DATAVALS, int,    "Total valid values"
	MISSVALS, int,    "Count of missing values"
	DATAMIN,  float,  "Minimum value"
	DATAMAX,  float,  "Maximum value"
	DATAMEAN, double, "Mean value"
	DATARMS,  double, "RMS standard deviation"
	DATASKEW, double, "Skewness of data distribution"
	DATAKURT, double, "Kurtosis of data distribution"

# the following should always be constant for the series
	WCSNAME,  string, "Carrington Heliographic/Time", "WCS name"
	WCSAXES,  int,    3,      "Number of WCS axes"
	CTYPE3,   string, "TIME", "WCS axis type"

(RSunRef will always be set to a constant value dependent on a compile definition of mtrack; the default is 696.0. Likewise, MDI_PA_Corr will be set to a compile-time constant iff mtrack is run with the -M flag; the default is -0.02)

The following key may occurr in data series in this class, and if present will be appropriately set by mtrack from the optional input parameter mai:

	MAI,      float,  "Magnetic Activity Index"

The following keys may occurr in data series in this class, and if present will be propagated by mtrack from the last record in the input data set:

	CONTENT,  string, "Observable"

If the special keyword PrimeKeyString is present in a data series in this class, its value will be set to the concatenated string of the values of the following keywords, provided that they are all present in the series:

	CarrTime  LonHG     LatHG     Duration  Width	  Height
	Source    ZonalTrk  MeridTrk  MapProj   MapScale

Sample series:
		xxx.noinfo
	This is a data series with the minimum structure necessary to hold the
	output of mtrack. It contains no keywords at all, so is of
	limited use in being able to retrieve records at a later time.
	Nevertheless, since it contains a data segment of required structure,
	it could be used to store the output in SUMS and retrieve the locations
	by using, for example, show_info -c and show_info -p n=...

	(To create the corresponding output series, clip the text between the
	**CLIP** lines, save it to a file named for example xxx.noinfo.jsd,
	and edit at least the Seriesname value to be a series in your DRMS
	namespace, e.g. su_name.noinfo

********	********	**CLIP**	********	********
Seriesname:	xxx.noinfo
Description:	"tracked mapped data cubes"
Author:		"My Name"
Archive:	0
Unitsize:	1
Retention:	10
Tapegroup:	0
Version:	2.1

#=====Segments=====
Data: V, vardim, short, 3, *, *, *, m/s, fits, compress, 0.0, 1.0, "Tracked Data Cube"
********	********	**CLIP**	********	********

		xxx.track
	This is a data series with the minimum number of prime keys necessary
	to isolate output cubes by time and location, and the WCS keywords that
	should be sufficient for proper geometric interpretation of the FITS
	data cubes. It also contains a segment named Log, to which the module's
	stdout will be copied if the module is run with the verbose flag -v

	Note that the last six keywords are constant for the series, implying
	that the DRMS does not store values of them in individual record rows.
	Those keywords, if present, must have the values shown, as those are
	the values always assigned by mtrack to the corresponding keywords if
	present; it is an error to assign a value to a constant keyword that
	differs from its constant value. (Of course they could be described as
	variable in the series, in which case mtrack would never complain, but
	it would waste space in the database table.)

	Note also that only the central location of each tracked cube in time and
	space is described by the collection of prime keys. If the same region
	were subsequently tracked with different parameters, for example size,
	duration, scale, map projection, or tracking rate, it would be in effect
	replace the previous record as an updated "version" according to the DRMS
	prime key logic.

********	********	**CLIP**	********	********
Seriesname:	xxx.track
Description:	"tracked mapped data cubes"
Author:		"My Name"
Archive:	0
Unitsize:	1
Retention:	10
Tapegroup:	0
PrimeKeys:	MidTime, LonHG, LatHG
Version:	2.1

#=====Keywords=====
Keyword: MidTime, time,   variable, record, NaN, 0, TAI, "Midpoint of tracking interval"
Keyword: LonHG,   float,  variable, record, NaN, %.1f, deg, "Longitude of tracked region center"
Keyword: LatHG,   float,  variable, record, NaN, %.1f, deg, "Latitude of tracked region center"
Keyword: CTYPE1,  string, variable, record, "?", %s, NA, "WCS axis type"
Keyword: CTYPE2,  string, variable, record, "?", %s, NA, "WCS axis type"
Keyword: CRPIX1,  float,  variable, record, NaN, %.1f, NA, "WCS reference pixel location"
Keyword: CRPIX2,  float,  variable, record, NaN, %.1f, NA, "WCS reference pixel location"
Keyword: CRPIX3,  float,  variable, record, NaN, %.1f, NA, "WCS reference pixel location"
Keyword: CRVAL1,  float,  variable, record, NaN, %.1f, NA, "WCS reference point value"
Keyword: CRVAL2,  float,  variable, record, NaN, %.1f, NA, "WCS reference point value"
Keyword: CRVAL3,  double, variable, record, NaN, %23.16e, sec, "WCS reference point value: Time since 1977.0"
Keyword: CDELT1,  float,  variable, record, NaN, %.4f, deg, "WCS coordinate increment at reference point"
Keyword: CDELT2,  float,  variable, record, NaN, %.7f, NA, "WCS coordinate increment at reference point"
Keyword: CDELT3,  float,  variable, record, NaN, %.7f, NA, "WCS coordinate increment at reference point"
# remainder are constant for series
Keyword: WCSNAME, string, constant, record, "Carrington Heliographic/Time", %s, NA, "WCS name"
Keyword: WCSAXES, int,    constant, record, 3, %d, NA, "WCS axes"
Keyword: CTYPE3,  string, constant, record, "TIME",%s, NA, "WCS axis type"
Keyword: CUNIT1,  string, constant, record, "deg", %s, NA, "WCS axis 1 units"
Keyword: CUNIT2,  string, constant, record, "deg", %s, NA, "WCS axis 2 units"
Keyword: CUNIT3,  string, constant, record, "s",   %s, NA, "WCS axis 3 units"

#=====Segments=====
Data: V,   vardim, short, 3, *, *, *, m/s, fits, compress, 0.0, 1.0, "Tracked Data Cube"
Data: Log, vardim, char, 0, NA, generic, "Tracking log"
********	********	**CLIP**	********	********

HMI Ring Diagrams

22 Nov 2021, 14:56-0800