Design RFC
This document outlines the design and implementation of the Ephmeris Abstraction Library. The USGS Astrogeology Science Center is looking for community comments and feedback on everything in this document. All of these can be made on the RFC feedback issue.
Thank for taking the time to read this! We hope that through your cooperation and feedback, EAL can be beneficial to the broader planetary science community.
-- ASC Software Team
The Ephemeris Abstraction Library (EAL) aims to solve two problems when dealing with ephemerides:
There are several different formats for storing ephemeris data. The most common is SPICE kernels, but this data is often accessed and then stored in a new format before calculations are done. These intermediate formats are opaque and confirming that everything is being properly converted between platforms can be laborious. Similar to how the GDAL project has solved this problem for image data formats, EAL hopes to solve this problem for ephemeris data formats.
The first targeted data formats are
- SPICE kernels
- ISIS cubes
- USGS CSM ISD meta-data files
We expect this list to grow as other data types are requested by the community.
Once ephemeris data is available to actually use it for calculations, users need to be able to interpolate over discrete data points. Often times, users will also want to fit functions over a data set and evaluate these functions. In C++, these operations involve various libraries such as GSL and Eigen, or even hand written code to perform different numerical calculations. EAL hopes to provide a single library that can be used to easily perform these calculations on ephemeris data using several different techniques that are chosen specifically for ephemeris data.
For interpolating positional data, the first targeted interpolation methods are:
- Piecewise linear
- Natural cubic spline
- Hermite cubic spline
For interpolating rotational data, the first targeted interpolation methods are:
- Piecewise linear quaternion interpolation
- Piecewise spherical quaternion interpolation
- Piecewise linear Euler angle interpolation
- Natural cubic spline interpolation
EAL will also support polynomial fitting and evaluation of positional data and Euler angles.
As with data formats, we expect these lists to change as the community requests support for new interpolation methods and functions.
EAL provides a C-Style C++ interface for users. The design goals for this interface are as follows:
- Minimize the number of methods and classes users need to work with
- Make it clear what calculations are being done with the data
- Make it easy to change between calculation methods
The current EAL API can be found at eal.h.
- Determine which SPICE kernels to load
- Query all of the required information from the SPICE kernels (make the spiceypy calls) The above two things are scary for ISIS integration. Take 1 week to determine if they really are. This functionality exists in pfeffernusse to some extent.
- JSON file I/O
- PDS3 I/O (maybe PDS4)
- spiceinit'd Cube I/O (InstrumentPosition, InstrumentRotation, BodyRotation, SunPosition)
- USGS CSM meta-data files
Different ephemeris data formats store different types of ephemeris data. For example, a USGS CSM ISD meta-data file has discrete rotation quaternions at different times in the image. A Bundle Adjusted ISIS3 cube on the other hand has polynomial functions that define Euler angle rotations through-out the image. In order to surface all of this data through a clean API, EAL returns the data as a JSON object. This way, there is a single end-point that users can access ephemeris data through. Then, the user can interrogate the JSON object and see exactly what was in the file.
In order to do this cleanly, EAL uses the JSON for Modern C++ library. It is a header only library that will be included with the EAL headers. From our experience using it in other projects, this is a powerful and easy to use library that makes working with JSON in C++ simple.
EAL plans to interpolate four different values:
- Position
- Velocity
- Rotation between two reference frames
- Angular Velocity between two reference frames
Each of these values is available via a different method that takes the following:
- Input data and times
- A time to interpolate at
- An interpolation method For position and velocity, these return the (x, y, z) vector. For rotation and angular velocity, these return the quaternion (w, x, y, z) vector.
EAL will support two different uses for function fitting. First, Bundle Adjustment requires functions in order to compute derivatives and apply updates. For this, EAL will allow for polynomial function fitting over position and rotations (in Euler angles). Second, data reduction is often helpful when working with large pushbroom images. For this purpose, EAL will support spline fitting and data reduction within an input tolerance.
EAL is currently under exploratory implementation, so this section is highly likely to change.
EAL consists of two components: a C++ library with a C-Style API to handle calculations and a Python module to handle file I/O and parsing. This design arose from concerns about performing complex file I/O and parsing in C++ and the issues with performing time sensitive calculations through a C++/Python interface. By splitting the two goals across two different languages, we can choose the languages that are best for meeting each goal. This adds some complexity at the interface between the two, but through some careful design choices, that can be kept to a minimum.
As an abstraction library, EAL combines many different libraries that perform complimentary tasks together. This section describes what third party libraries EAL uses "under the hood" and why they were chosen.
EAL uses Anaconda to handle third party dependencies. At the USGS Astrogeology Science Center (ASC), we've found that Anaconda provides an easy way to create contained build environments for our software. It also provides for an easy release and distribution mechanism. Check out the ASC Anaconda cloud page to see all of the packages we distribute over Anaconda.
EAL uses the GNU Scientific Library (GSL) to evaluate polynomial functions, perform linear interpolation, and perform natural cubic spline interpolation. We chose GSL because it is a library that we are familiar with and it is a widely used C library for working with scientific data. As a C library it is very fast, which is important for the success of EAL.
EAL uses Eigen for quaternion interpolation and calculations.
As a C++ library, EAL is unusual in that it consumes a Python module. Normally, Python modules consume C/C++ libraries. We chose to take this unusual step for one main reason, handling file I/O and parsing in C++ is a serious pain. Python is an excellent alternative for EAL. There are a large number of python modules for working with different planetary data formats that are not available in C++. It also makes developing interfaces to new file formats easier and quicker.
The biggest issue with using Python is its speed. For this reason, we isolated the interface with Python to file I/O which is a one time operation and not critical to the overall runtime of most use cases. The computational component of EAL is intentionally not written in Python for this exact reason.
Another issue with calling Python from C++ is that there needs to be either an interface library or calls directly to Python's C API wherever it happens. In EAL, there is only place where the C++ and Python code interface. This is a C++ method that calls a python method to parse a file and return a JSON encoded data string. That is the only place where the Python and C++ code interface.
We chose JSON as our data abstraction format for two reasons. First, it allows for a single interface between the Python and C++ code. By having a flexible, transparent data format, the Python parsing code only has to honor a limited number of requirements and the C++ code can easily check for them. Second, JSON's nested structure allows both complex and simple ephemeris data formats to return the same thing. For example, an ISIS3 cube with multiple positions and rotations can return a nested JSON object and a SPICE rotation CK can return a single rotation data object. In both cases, the user interface is the same.