Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Draft] Run with DINO data #30

Open
wants to merge 25 commits into
base: main
Choose a base branch
from
Open

[Draft] Run with DINO data #30

wants to merge 25 commits into from

Conversation

ma595
Copy link
Member

@ma595 ma595 commented Jan 29, 2025

This PR generalises the pipeline to work with DINO data.

  • Resamples DINO monthly ssh output to annual.
  • Adds a description of the pipeline in the README.md for generating a ML projected restart file (compatible with DINO).
  • Amends hardcoded path in main_restart.py
  • Modifies lib/restart.py to ensure compatibility with ML generated DINO files.
  • Simple restart toolbox script Create simple script for restart toolbox #33
  • Modify main_forecast.py so that it's compatible with DINO data and add docstring / improve structure.
  • Make forecast deterministic given some random seed.

Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@ma595
Copy link
Member Author

ma595 commented Jan 29, 2025

Hi @isaacaka, a few comments:
Can you please rebase on top of changes to main.
Also, the image you provide in step 9 indicates 100 years. Have you managed to get to this point yet?

I've made a few changes in PR#31.

Let me know if you have any questions about fixing the conflicts.

@ma595
Copy link
Member Author

ma595 commented Feb 18, 2025

For reference, my env.sh looks like this:

#! /bin/bash
# path to simus_predicted
# path to REBUILD_NEMO tools.
# path to DINO/NEMO simulation output

# CSD3 specific modules
module purge
module load rhel8/default-ccl
module load python/3.11.9/gcc/nptrdpll
module load netcdf-fortran/4.6.1/intel/intel-oneapi-mpi/kqukipdf
module load boost/1.85.0/gcc/zouxm6hy
export PYTHONENV=./venv3.9/bin/activate

source $PYTHONENV
# NEMO + XIOS
NEMO=/rds/project/rds-5mCMIDBOkPU/ma595/nemo/NEMO/
NEMO_PATH=$NEMO/nemo_4.2.1
XIOS=$NEMO/trunk
export CPATH=$NEMO/trunk/inc:$CPATH
export LD_LIBRARY_PATH=$XIOS/lib:$LD_LIBRARY_PATH
export REBUILD_NEMO=${NEMO_PATH}/tools/REBUILD_NEMO/rebuild_nemo


# output
output=$NEMO_PATH/tests/DINO
# point to ml output
ml_prediction_dir=$output/concatenate_grid_150/simus_predicted
# point to nemo output, checkpoint
nemo_output_dir=/rds/project/rds-5mCMIDBOkPU/ma595/nemo/NEMO/nemo_4.2.1/tests/DINO/EXP00-150years-restart-identity/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants