Skip to content

Commit

Permalink
Merge pull request #17 from euroargodev/refactoring-as-a-clean-module…
Browse files Browse the repository at this point in the history 
…-and-cli

Refactoring with clean CLI and python interfaces
  • Loading branch information
@gmaze
gmaze authored Apr 12, 2024
2 parents abef979 + 35c9617 commit 5c2c59c
Show file tree
Hide file tree
Showing 53 changed files with 4,110 additions and 2,215 deletions.
5 changes: 4 additions & 1 deletion .gitignore
View file
Original file line number Diff line number Diff line change
Expand Up @@ -142,4 +142,7 @@ cli/build-pypi
*.nc
*.npi*
cli/vfrecov/
webapi/myapp/static/data
webapi/myapp/static/data
vfrecovery_simulations_data/
vfrecovery/static/assets/simulations_registry.pkl
vfrecovery/static/assets/simulations_registry.pkl
234 changes: 150 additions & 84 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,112 +2,178 @@
|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|
| [![DOI](https://zenodo.org/badge/543618989.svg)](https://zenodo.org/badge/latestdoi/543618989) |

The goal of this repository is to provide a library to make Argo floats trajectory predictions easy, in order to facilitate recovery.
The library produces a prediction _patch_ or _cone_ that could be displayed on a map like here: https://floatrecovery.euro-argo.eu
More about Argo floats recovery in here: https://github.com/euroargodev/recovery/issues
The goal of this repository is to provide a CLI and Python library to make Argo floats trajectory predictions easy, in order to facilitate recovery.

New version compatible with [VirtualFleet 0.4.0](https://virtualfleet.readthedocs.io/en/latest/whats-new.html#v0-4-0-2-feb-2024) and using the [Copernicus Marine Toolbox](https://help.marine.copernicus.eu/en/collections/5821001-python-library-api) to retrieve GLORYS or ARMOR3D velocity fields.
More about Argo floats recovery in here:
- https://floatrecovery.euro-argo.eu
- https://github.com/euroargodev/recovery/issues

# Documentation (preliminary)

## Overall prediction procedure
1. Given a specific float cycle to predict ``C``, we extract:
- space/time position of the previous cycle ``C-1``,
- configuration parameters of the previous cycle ``C-1``, such as parking depth, profiling depth and cycling period using the EA API (but these can be overwritten if necessary).
# Documentation

2. We download the daily CMEMS velocity fields for a region around the previous cycle ``C-1`` coordinates
## Command Line Interface

3. We run a VirtualFleet simulation:
- where we use a large number of virtual floats located with a random perturbations around the float cycle ``C-1`` position in space/time
- for the cycle ``C-1`` duration
Primary groups of commands are ``predict``, ``describe`` and ``db``.

4. We compute the most probable position of the float cycle ``C`` and prediction metrics and figures.
### vfrecovery predict
```
Usage: vfrecovery predict [OPTIONS] WMO CYC
Execute the VirtualFleet-Recovery predictor
WMO is the float World Meteorological Organisation number.
CYC is the cycle number location to predict. If you want to simulate more
than 1 cycle, use the `n_predictions` option (see below).
Options:
-v, --velocity TEXT Velocity field to use. Possible values are:
'GLORYS', 'ARMOR3D' [default: GLORYS]
--output_path TEXT Simulation data output folder [default:
'./vfrecovery_simulations_data/<WMO>/<CYC>']
--cfg_parking_depth FLOAT Virtual floats parking depth in db [default:
previous cycle value]
--cfg_cycle_duration FLOAT Virtual floats cycle duration in hours
[default: previous cycle value]
--cfg_profile_depth FLOAT Virtual floats profile depth in db [default:
previous cycle value]
--cfg_free_surface_drift INTEGER
Virtual cycle number to start free surface
drift, inclusive [default: 9999]
-np, --n_predictions INTEGER Number of profiles to predict after cycle
specified with argument 'CYC' [default: 0]
-nf, --n_floats INTEGER Swarm size, i.e. the number of virtual
floats simulated to make predictions
[default: 100]
-s, --domain_min_size FLOAT Minimal size (deg) of the simulation domain
around the initial float position [default:
5]
--overwrite Should past simulation data be overwritten
or not, for a similar set of arguments
--lazy / --no-lazy Load velocity data in lazy mode (not saved
on file). [default: lazy]
--log_level [DEBUG|INFO|WARN|ERROR|CRITICAL|QUIET]
Set the details printed to console by the
command (based on standard logging library).
[default: INFO]
-h, --help Show this message and exit.
Examples:
vfrecovery predict 6903091 112
```

### vfrecovery describe

The reason why we make random perturbations of the float cycle ``C-1`` position is not because the float position is uncertain (with GPS it is fairly accurate most of the time), but because it is a cheap way to account for errors in the velocity field. Indeed, we assume that the _phase_ of the velocity field used to advect floats is the primary source of uncertainties to predict the final position. We do not account for velocity shear/strain errors at this point.
```
Usage: vfrecovery describe [OPTIONS] TARGET WMO [CYC]...
## Installation
TARGET select what is to be described. A string in: ['obs', 'velocity',
'run'].
Our goal is to distribute VFrecovery as a standalone pipy package. In the meantime, one need to work with this repo only.
WMO is the float World Meteorological Organisation number
- Download his repository:
```bash
git clone [email protected]:euroargodev/VirtualFleet_recovery.git
```
- Add the ``cli`` folder to your path, eg:
```bash
export PATH="/Users/gmaze/git/github/euroargodev/VirtualFleet_recovery/cli:$PATH"
```
- Make sure to get the appropriate Python 3.9 environment ([using this conda file](environment.yml)):
```bash
mamba env create -f environment.yml
```
- Install the last VirtualFleet version:
```bash
git clone [email protected]:euroargodev/VirtualFleet.git
```
CYC is the cycle number location to restrict description to
## Command line instructions
Options:
--log-level [DEBUG|INFO|WARN|ERROR|CRITICAL|QUIET]
Set the details printed to console by the
command (based on standard logging library).
[default: INFO]
-h, --help Show this message and exit.
### Usage
The ``recovery_prediction.py`` script allows making predictions, i.e. at this point to produce:
- a json data files with predictions information for machine/machine applications,
- and a few figures to indicate where the float will make surface contact and how the probability patch was created.
Examples:
vfrecovery describe velocity 6903091
vfrecovery describe obs 6903091 112
```

### vfrecovery db

For a simple help, you can type:
```
recovery_prediction.py -h
```
Usage: vfrecovery db [OPTIONS] ACTION
Internal simulation database helper
Options:
--log-level [DEBUG|INFO|WARN|ERROR|CRITICAL|QUIET]
Set the details printed to console by the
command (based on standard logging library).
[default: INFO]
-i, --index INTEGER Record index to work with
-h, --help Show this message and exit.
Examples:
To make prediction of where the 99th cycle of the 6902919 float will be, just type:
vfrecovery db info
vfrecovery db read
vfrecovery db read --index 3
vfrecovery db drop
```
recovery_prediction.py 6902919 99


## Python interface


### vfrecovery.predict

```python
import vfrecovery

wmo, cyc = 6903091, 126
results = vfrecovery.predict(wmo, cyc)
```

A few options are available:
Signature:
```
usage: recovery_prediction.py [-h] [--nfloats NFLOATS] [--output OUTPUT] [--velocity VELOCITY] [--save_figure SAVE_FIGURE] [--save_sim SAVE_SIM] [--vf VF] [--json]
[--cfg_parking_depth CFG_PARKING_DEPTH] [--cfg_cycle_duration CFG_CYCLE_DURATION]
wmo cyc
VirtualFleet recovery predictor
positional arguments:
wmo Float WMO number
cyc Cycle number to predict
optional arguments:
-h, --help show this help message and exit
--nfloats NFLOATS Number of virtual floats used to make the prediction, default: 2000
--output OUTPUT Output folder, default: webAPI internal folder
--velocity VELOCITY Velocity field to use. Possible values are: 'ARMOR3D' (default), 'GLORYS'
--save_figure SAVE_FIGURE
Should we save figure on file or not ? Default: True
--save_sim SAVE_SIM Should we save the simulation on file or not ? Default: False
--vf VF Parent folder to the VirtualFleet repository clone
--json Use to only return a json file and stay quiet
--cfg_parking_depth CFG_PARKING_DEPTH
Virtual floats parking depth in [db], default: use previous cycle value
--cfg_cycle_duration CFG_CYCLE_DURATION
Virtual floats cycle duration in [hours], default: use previous cycle value
This script can be used to make prediction of a specific float cycle position.
This script can be used on past or unknown float cycles.
Note that in order to download online velocity fields from the Copernicus Marine Data Store, you need to have the
appropriate credentials file setup.
(c) Argo-France/Ifremer/LOPS, 2022-2024
vfrecovery.predict(
wmo: int,
cyc: int,
velocity: str = 'GLORYS',
output_path: Union[str, pathlib.Path] = None,
n_predictions: int = 0,
cfg_parking_depth: float = None,
cfg_cycle_duration: float = None,
cfg_profile_depth: float = None,
cfg_free_surface_drift: int = 9999,
n_floats: int = 100,
domain_min_size: float = 5.0,
overwrite: bool = False,
lazy: bool = True,
log_level: str = 'INFO',
)
```

So, don't forget to:
- set up your environment to be able to download velocity fields from the Copernicus Marine Toolbox
- use the option ``vf`` to specify where the VirtualFleet software has been cloned (this is temporary and will change once VirtualFleet will be available on Pypi).

### Example

# API Design

## Other possible commands

```bash
vfrecovery meetwith "cruise_track.csv" WMO CYC0
```

## Data storage
Simulation data are stored on disk under the following architecture:

```
recovery_prediction.py 6902915 116
./vfrecovery_simulations_data
|- vfrecovery_simulations.log
|- WMO
|----CYC
|----VELOCITY(NAME + DOWNLOAD_DATE + DOMAIN_SIZE)
|- velocity_file.nc
|- figure.png
|---- RUN_PARAMS(NP + CFG + NF)
|- float_configuration.json
|- trajectories.zarr
|- results.json
|- figure.png
```
Below is an example of this prediction for the 99th cycle of the 6902919 float.
The really observed 99th cycle is shown at the tip of the arrow (red point) starting from the previous 98th cycle.
The VirtualFleet Recovery prediction is in the probabilistic red shading: the most probable position predicted is in the redder region.
![Figure](docs/img/vfrecov_predictions_recap_VELARMOR3D_NF2000_CYCDUR240_PDPTH1000.png)

This ensures that for a given velocity field, all possible simulations are unambiguously found under a single folder
47 changes: 0 additions & 47 deletions cli/launch_webapi
View file

This file was deleted.

Loading

0 comments on commit 5c2c59c

Please sign in to comment.