Skip to content

Check if MOM6 can reproduce across restarts

Gustavo Marques edited this page Nov 28, 2017 · 2 revisions

Part of our testing protocol consists of checking if MOM6 can reproduce answers across restarts. This page provides a description on how to perform such test using the CESM framework.

1) Create a new case

First, create a new case following this instructions. Make sure to check the branch that needs to be tested before building the case.

2) Change parameters using xmlchange

Run the following commands on your $CASEROOT:

./xmlchange REST_OPTION=ndays
./xmlchange REST_N=1 
./xmlchange STOP_OPTION=ndays
./xmlchange STOP_N=2
./xmlchange DEBUG=TRUE
./xmlchange DOUT_S=FALSE

This will run the model for 2 days and save a restart file every day. Although not always necessary, it is a good ideal to set DEBUG=TRUE in case you also need to turn on this option in MOM6 (see #6).

3) COLD run

Submit the COLD run:

./case.submit

Once the run has finished, go to $RUNDIR and do the following:

cp ocean.stats ocean.stats.COLD
cp cesm.log.* cesm.log.COLD.gz

4) RESTART run

On $RUNDIR, type the following:

sed -i 's/0001-01-03/0001-01-02/g' rpointer.*

This will point the models to the restart files created at the end of day one.

Go to $CASEROOT and run the following:

./xmlchange CONTINUE_RUN=TRUE
./xmlchange STOP_N=1

Submit the RESTART run:

./case.submit

Once the run has finished, go to $RUNDIR and do the following:

cp ocean.stats ocean.stats.RESTART
cp cesm.log.* cesm.log.RESTART.gz

5) Comparing COLD/RESTART

Go to $RUNDIR and compare, between days 1 and 2, ocean.stats.COLD versus ocean.stats.RESTART as well as cesm.log.COLD.gz versus cesm.log.RESTART.gz. With exception of Me, Se and Te at day 1 (in ocean.stats), the results should be bit-wise identical.

6) If the case does not reproduce across restarts

In case the results are not bit-wise identical, the first step is to identify where the code the answers are diverging. To do so, go to $RUNDIR and add #override DEBUG=True in MOM_override. Repeat steps #3 and #4 and compare cesm.log.COLD.gz versus cesm.log.RESTART.gz. You should be able to see where in the code the answers are diverging.