solar2:/web/hmi/htdocs/development/jim/dds_soc.straw 08May2003 DDS - HMI SOC Strawman Data Flow -------------------------------- 1. The DDS will put and get files on the HMI SOC via an ftp or similar type program (e.g. FASTCopy). 2. The normal SOC directories for file interchange are: /dds2soc /soc2dds 3. Every minute the DDS writes to /dds2soc a telemetry and quality and accounting file named like so and in this order: HMI_2002.11.14_10:31:00.tlm HMI_2002.11.14_10:31:00.txt 4. The .tlm file will be ~400MB and contain HMI VCDUs in time order and without duplication. Real-time delivery will be attempted only once by the DDS as files become available. 5. The .txt file will be an ascii file with TBD quality and accounting information. Two item of information will the name and size in bytes of the corresponding tlm file. The .txt contains an end of file indicator field. 6. The SOC will initiate processing the .tlm file when the proper .txt is received and verified (txt size, tlm name, tlm size, TBD...). 7. Once an hour on the hour the DDS writes to /dds2soc a delivery status file (dsf) named like so: HMI_2002.11.14_10:00:00.dsf This .dsf contains the file names of all the telemetry files that the DDS has that have not yet been acknowledged by the SOC. The dsf file will contain ASCII text entries as follow: , , will be formatted as shown above. will be the size of the file in bytes. will be a numerical filed that can have one of three values: 0 - Delivery Pending; File has not been delivered for whatever reason 1 - Delivered; DDS thinks file has been delivered, waiting for SOC ack 2 - Retransmit; SOC requested a retransmit, retransmit qued 3 - Retransmit Failed 4 - Retransmit Successful 8. Upon receipt of the .dsf the SOC will check all the Delivery Pending and Delivered files against what it has successfully received and write an acknowledgement status file (asf) in /soc2dds. The asf file will contain ASCII text entries identical in format to that of the dsf file. The field will have one of 4 values: 0 - Delivery Pending; File has not been delivered for whatever reason 1 - N/A 2 - Retransmit; SOC requested a retransmit 3 - SOC Acknowledged, SOC acknowledging receipt of file Files marked for retransmission may have 2 optional fields: Machine to which retransmissions are to be delivered, must be know by the DDS Must be writeable by the DDS If optional fields are absent, data will be delivered to default location. Files marked for retransmission will be delivered as bandwidth allows. In general, any file name can be added to the asf for the SOC to request a retransmission. The only way that the SOC makes a tlm file retransmission request is via this /soc2dds .asf file. The DDS removes the /soc2dds .asf file after it retrieves it. 9. At the end of each day the SOC will write a .ack file to /soc2dds containing the tlm file names that have been archived successfully on that day. The DDS will not delete a file that has not been so acknowledged but will notify the SOC manager by e-mail that an acknowledgement is still pending. This is done at say 20 days, well before the norminal 30day limit to delete the file. (The 30d delete should not be hardwired. If there is a real problem and disk space and lots of phone calls, the data should not be dropped due to rigidity in cron scripts or the like.) In summary here are the file types that are exchanged: Extention Originator Frequency Content --------- ---------- --------- ------- .tlm DDS 1minute HMI VCDUs .txt DDS 1minute Ascii quality and accounting information .dsf DDS 1hour DDS delivery status file .asf SOC 1hour SOC acknowledgement status file .ack SOC 1day tlm file names succesfully archived ----------------------------------------------------------------------------- Footnote: We will definitively know all the HMI telemetry to expect from interpretation of the hk data. Eventually this will be defined to be the definitive means to know what files to expect from the DDS.