ICS and QLF interface

ICS will provide a python module called ICSInterface to query the exposureDB.

We have three methods, the first one to return summary information for the latest exposure or for a range of exposures:

ICSInterface.get_exposure_summary(self, date_range=None, expid_range=None, require_data_written=True)
        """
        Input Arguments:
            date_range - tuple with low and high datetime object. Timestamp is when data_written was updated. if "last" instead of tuple, only return newest
            expid_range - tuple with low and high. If one is None, only use one expid. if "last" instead of tuple, only return newest
            require_data_written - default True. If True, only return exposures where all data is on disk
        Output:
            list of dictionaries with exposure summary. Might be empty.
        Raise exception if no range given, or if two ranges given
        """

this method returns a dictionary with the following keys (corresponding to the database column names, not necessarily identical to the FITS header keys):

- expid
- skyra (telra)
- skydec (teldec) 
- exptime 
- program (dark, grey, bright)
- tile_id (tile) 
- exptype (or flavor?) 
- date (dateobs) (UTC)
- night (YYYYMMDD) new field in the database, use that to have the same definition of night.
- seeing
- airmass
- transparency
- skylevel
- s2n
- data_written (flag defining that the data is ready to be used)
- data_location (ICS file system path, it is not decided yet if it will be readable by other subsystems)

there's another method to get the files associated to an exposure, the destination can be provided by us (then ICS does the copy) or we use the file_path returned to execute the copy (e.g, rsync) on our side.

ICSInterface.get_exposure_files(self, expid, dest=None, file_class = ['desi', 'fibermap'], overwrite=True)
        """
        Input Arguments:
            expid (required)
            dest (optional, directory to copy data to, if given)
            file_class (list of file classes to copy can be desi, guider, acquisition, sky, fvc, focus or fibermap)
            overwrite (default True; if True, and destination, copy files on top of existing)
        Output:
            file_path (either ours on NFS or dest after copy)
        Exception if data_written boolean in exposure database is False
        All other errors also raise exception
        """

List of files returned, the class used to identify the file is also used as the file prefix.

desi-EXPID.fits.fz - MEF FITS file with the 30 CCD extension and a table with spectrograph data.

fibermap-EXPID.fits - FITS table that includes information from the input tile file we get from the NFS and includes the fiber positioner data from the ICS (e.g. your x_target, y_target, x_fvcobs, y_fvcobs etc)

NOTE: we looked at a file the QL(F) team uses that comes from some simulation. It appears that some information is duplicated and Kyle will go through this and come up with a format for both the tile file and this fibermap file. One request I expressed was to rename x_fvcobs to x_obs (keep fvc out of this because you want the coordinates in the focal plane, not the fvc pixel plane)

guider-EXPID.fits.fz - MEF FITS file with all guider (full frame) exposures from the 6 guider GFAs for this exposure.

focus-EXPID.fits.fz - MEF FITS file for the images from the 4 focus GFAs. As for the guider, the number of extensions, i.e. GFA exposures per desi exposure will be variable.

sky-EXPID.fits.fz - MEF FITS file with all sky camera (full frame or centroid information - tbd by David Kirkby) for this exposure.

etc-EXPID.fits - FITS table with the ETC outputs (s/n, seeing, transparency, skylevel) during this desi exposure (the number of rows in this table should be the same as the number of sky camera exposures).

acquisition-EXPID.fits - MEF FITS file with the acquisition image from the 6 guider GFAs (the one at the beginning of the positionner loop we use to do the astrometry). We keep the option open to reaqcuire in which case

(*) We need to add these files to the datamodel, probably extending the gfa-EXPID description.

NOTE: calibration files needed to run the QL pipeline are not provided by ICS, someone must generate and put them in the directory configured at qlf.cfg.

and a third method can be used to register alarms so that they are displayed at the ICS Observer Console:

ICSInterface.alarm(self, message, level='EVENT', **kwargs) 
    """ 
    Send an alarm to DOS
    Input Arguments:
        message (message sent do DOS)
        level ('EVENT', 'ALERT', 'WARNING', 'CRITICAL')
    The message will be recorded in the alarm database and the observer will be notified if an instance is running
    The kwargs dictionary is passed on to the DOS alarm call and includes optional parameters such as id, logbook, email etc
    """