"Modify an ACCESS model's source code" doc needs an additional step in the spack develop instructions

[harshula]

Description

The Mark package as a development package section needs an additional step.

When spack develop is called, e.g. spack develop <package>@git.<gitref>=<spack-version>, the user must manually check the environment's spack.yaml file to see if there's an existing reference to the <package> in the specs or packages section. If there is, then it must be commented out to allow the concretize step to succeed.

Why

We start with environments/om2-develop/spack.yaml containing:

  packages:
...
    mom5:
      require:
        - '@git.2023.11.09'

[om2-develop] [root@42f468be1309 opt]# spack develop [email protected]_2024.08.14=master

A new section has been added to environments/om2-develop/spack.yaml and it conflicts with the the packages section:

  develop:
    mom5:
      spec: [email protected]_2024.08.14=master

[om2-develop] [root@42f468be1309 opt]# spack concretize -f
==> Error: Spack concretizer internal error. Please submit a bug report and include the command, environment if applicable and the following error message.
    [email protected]=2024.03.0 is unsatisfiable, errors are:
    internal_error("Reuse version weight used for build package")
    internal_error("must choose a single version to satisfy version constraints")
    internal_error("version weights must exist and be unique")

[om2-develop] [root@42f468be1309 opt]# spack concretize -f --fresh
==> Error: concretization failed for the following reasons:

   1. cannot satisfy a requirement for package 'mom5'.

Comment out the conflicting section in environments/om2-develop/spack.yaml:

#    mom5:
#      require:
#      - '@git.2023.11.09'

Then the concretize step with --fresh or --reuse-deps succeeds:

[om2-develop] [root@42f468be1309 opt]# spack concretize -f
==> Error: Spack concretizer internal error. Please submit a bug report and include the command, environment if applicable and the following error message.
    [email protected]=2024.03.0 is unsatisfiable, errors are:
    internal_error("Reuse version weight used for build package")
    internal_error("must choose a single version to satisfy version constraints")
    internal_error("version weights must exist and be unique")

[om2-develop] [root@42f468be1309 opt]# spack concretize -f --fresh
==> Concretized [email protected]=2024.03.0
...

[om2-develop] [root@42f468be1309 opt]# spack concretize -f --reuse-deps
==> Concretized [email protected]=2024.03.0
...

[harshula]

We should also consider changing the instructions to use --reuse-deps instead of --fresh. It's much quicker because existing dependency packages are not rebuilt.

https://github.com/spack/spack/releases/tag/v0.22.0:

With the install command you now have three options:

    --reuse (default): reuse as many existing installations as possible.
    --reuse-deps / --fresh-roots: upgrade (freshen) roots but reuse dependencies if possible.
    --fresh: install fresh versions of requested packages (roots) and their dependencies.

We've also introduced --fresh-roots as an alias for --reuse-deps to make it more clear
that it may give you fresh versions. More details in #41302 and #43988.

--fresh

[om2-develop] [root@42f468be1309 opt]# spack concretize -f --fresh
==> Concretized [email protected]=2024.03.0
 -   jzwwfpn  [email protected]=2024.03.0%[email protected]~deterministic build_system=bundle arch=linux-rocky8-x86_64
 -   5eptl6x      ^[email protected]=2023.10.19%[email protected]~deterministic~optimisation_report build_system=makefile arch=linux-rocky8-x86_64
[+]  ctrofap          ^[email protected]%[email protected] build_system=autotools patches=80b9577 arch=linux-rocky8-x86_64
[+]  k2uau2r          ^[email protected]%[email protected]~guile build_system=generic arch=linux-rocky8-x86_64
 -   tre5nn5          ^[email protected]%[email protected]~blosc~byterange~dap~fsync~hdf4~jna+mpi~nczarr_zip+optimize~parallel-netcdf+pic+shared~szip~zstd build_system=autotools arch=linux-rocky8-x86_64
 -   jv5qfgz              ^[email protected]%[email protected]~cxx~fortran+hl~ipo~java~map+mpi+shared~subfiling~szip~threadsafe+tools api=default build_system=cmake build_type=Release generator=make patches=82088c8 arch=linux-rocky8-x86_64
 -   7xvufuz          ^[email protected]%[email protected]~doc+pic+shared build_system=autotools patches=b050dbd arch=linux-rocky8-x86_64
 -   wxgmvsb          ^[email protected]=2023.11.09%[email protected]~deterministic~optimisation_report build_system=makefile arch=linux-rocky8-x86_64
 -   yrzgzaf          ^[email protected]%[email protected]~atomics~cuda~cxx~cxx_exceptions~gpfs~internal-hwloc~internal-libevent~internal-pmix~java~legacylaunchers~lustre~memchecker~openshmem~orterunprefix~romio+rsh~singularity~static+vt+wrapper-rpath build_system=autotools fabrics=none patches=073477a,60ce20b romio-filesystem=none schedulers=none arch=linux-rocky8-x86_64
 -   3rpq2un              ^[email protected]%[email protected]~cairo~cuda~gl~libudev+libxml2~netloc~nvml~oneapi-level-zero~opencl+pci~rocm build_system=autotools libs=shared,static arch=linux-rocky8-x86_64
 -   rvk6ysq                  ^[email protected]%[email protected] build_system=autotools arch=linux-rocky8-x86_64
 -   bnnnbif                      ^[email protected]%[email protected] build_system=autotools arch=linux-rocky8-x86_64
 -   xihdlwh                  ^[email protected]%[email protected]+pic~python+shared build_system=autotools arch=linux-rocky8-x86_64
 -   wt2wcjf                      ^[email protected]%[email protected] build_system=autotools libs=shared,static arch=linux-rocky8-x86_64
 -   bqcqoj3                      ^[email protected]%[email protected]~pic build_system=autotools libs=shared,static arch=linux-rocky8-x86_64
 -   ax3arpd              ^[email protected]%[email protected]+openssl build_system=autotools arch=linux-rocky8-x86_64
 -   4vblppx                  ^[email protected]%[email protected]~docs+shared build_system=generic certs=mozilla arch=linux-rocky8-x86_64
 -   qcyukwo                      ^ca-certificates-mozilla@2023-05-30%[email protected] build_system=generic arch=linux-rocky8-x86_64
 -   wc5kt2a              ^[email protected]%[email protected] build_system=autotools patches=4e1d78c,62fc8a8,ff37630 arch=linux-rocky8-x86_64
 -   u4vctlv                  ^[email protected]%[email protected] build_system=autotools arch=linux-rocky8-x86_64
 -   76fo36h                  ^[email protected]%[email protected] build_system=autotools arch=linux-rocky8-x86_64
 -   fq7xcox                  ^[email protected]%[email protected] build_system=autotools arch=linux-rocky8-x86_64
 -   43nbmx6                      ^[email protected]%[email protected] build_system=autotools patches=440b954 arch=linux-rocky8-x86_64
 -   e55wk22                  ^[email protected]%[email protected]+sigsegv build_system=autotools patches=9dc5fbd,bfdffa7 arch=linux-rocky8-x86_64
 -   m3x35pc                      ^[email protected]%[email protected] build_system=autotools arch=linux-rocky8-x86_64
 -   yqirtwl                      ^[email protected]%[email protected] build_system=autotools arch=linux-rocky8-x86_64
 -   ykp6bv7              ^[email protected]%[email protected]+gssapi build_system=autotools arch=linux-rocky8-x86_64
 -   gd46vh4                  ^[email protected]%[email protected]+shared build_system=autotools arch=linux-rocky8-x86_64
 -   5fgxqdz                      ^[email protected]%[email protected]~color build_system=autotools arch=linux-rocky8-x86_64
 -   3sqwvp6                      ^[email protected]%[email protected]+bzip2+curses+git~libunistring+libxml2+pic+shared+tar+xz build_system=autotools arch=linux-rocky8-x86_64
 -   37k3deq                          ^[email protected]%[email protected]~debug~pic+shared build_system=generic arch=linux-rocky8-x86_64
 -   2xemo3z                          ^[email protected]%[email protected] build_system=autotools zip=pigz arch=linux-rocky8-x86_64
 -   p7ppejr                              ^[email protected]%[email protected] build_system=makefile arch=linux-rocky8-x86_64
 -   haxkjxt                              ^[email protected]%[email protected]+programs build_system=makefile compression=none libs=shared,static arch=linux-rocky8-x86_64
 -   wr7hko2                  ^[email protected]%[email protected] build_system=autotools arch=linux-rocky8-x86_64
 -   z5ks2e6                  ^[email protected]%[email protected]~obsolete_api build_system=autotools patches=4885da3 arch=linux-rocky8-x86_64
[e]  3qsufzn              ^[email protected]%[email protected]+cpanm+opcode+open+shared+threads build_system=generic patches=8cf4302 arch=linux-rocky8-x86_64
 -   zj5tvyh              ^[email protected]%[email protected]~docs+pmi_backwards_compatibility~python~restful build_system=autotools arch=linux-rocky8-x86_64
 -   qm7p5m7              ^[email protected]%[email protected]+compat+new_strategies+opt+pic+shared build_system=autotools arch=linux-rocky8-x86_64
 -   jpzbsj4          ^[email protected]%[email protected]+fortran~ipo~logging+mpi~pnetcdf~shared~timing build_system=cmake build_type=Release generator=make patches=55a6d7a arch=linux-rocky8-x86_64
[e]  7sn3ltj      ^[email protected]%[email protected] build_system=autotools arch=linux-rocky8-x86_64
 -   bhweot2      ^[email protected]=2023.10.26%[email protected]~deterministic~ipo~optimisation_report build_system=cmake build_type=Release generator=make arch=linux-rocky8-x86_64
 -   4oekw7r          ^[email protected]%[email protected]~doc+ncurses+ownlibs build_system=generic build_type=Release arch=linux-rocky8-x86_64
 -   kgqdkph              ^[email protected]%[email protected]~gssapi~ldap~libidn2~librtmp~libssh~libssh2+nghttp2 build_system=autotools libs=shared,static tls=openssl arch=linux-rocky8-x86_64
 -   4a5empx                  ^[email protected]%[email protected] build_system=autotools arch=linux-rocky8-x86_64
 -   wt3dyrj              ^[email protected]%[email protected]~symlinks+termlib abi=none build_system=autotools patches=7a351bc arch=linux-rocky8-x86_64
 -   ckw5vy4          ^[email protected]%[email protected]~ipo build_system=cmake build_type=Release generator=make arch=linux-rocky8-x86_64
 -   fiyxxp5          ^[email protected]%[email protected] build_system=autotools arch=linux-rocky8-x86_64
 -   k6miipr      ^[email protected]_2024.08.14=master%[email protected]~access-gtracers~deterministic~optimisation_report+restart_repro build_system=makefile dev_path=/opt/environments/om2-develop/mom5 type=ACCESS-OM arch=linux-rocky8-x86_64

==> Updating view at /opt/environments/om2-develop/.spack-env/view

--reuse-deps

[om2-develop] [root@42f468be1309 opt]# spack concretize -f --reuse-deps
==> Concretized [email protected]=2024.03.0
 -   bev3rbt  [email protected]=2024.03.0%[email protected]~deterministic build_system=bundle arch=linux-rocky8-x86_64
[+]  54uxawk      ^[email protected]=2023.10.19%[email protected]~deterministic~optimisation_report build_system=makefile arch=linux-rocky8-x86_64
[+]  ctrofap          ^[email protected]%[email protected] build_system=autotools patches=80b9577 arch=linux-rocky8-x86_64
[+]  k2uau2r          ^[email protected]%[email protected]~guile build_system=generic arch=linux-rocky8-x86_64
[+]  3hib7t6          ^[email protected]%[email protected]~blosc~byterange~dap~fsync~hdf4~jna+mpi~nczarr_zip+optimize~parallel-netcdf+pic+shared~szip~zstd build_system=autotools arch=linux-rocky8-x86_64
[+]  a72goau              ^[email protected]%[email protected]~cxx~fortran+hl~ipo~java~map+mpi+shared~subfiling~szip~threadsafe+tools api=default build_system=cmake build_type=Release generator=make patches=82088c8 arch=linux-rocky8-x86_64
[+]  kzbn4t4          ^[email protected]%[email protected]~doc+pic+shared build_system=autotools patches=b050dbd arch=linux-rocky8-x86_64
[+]  ruyz6py          ^[email protected]=2023.11.09%[email protected]~deterministic~optimisation_report build_system=makefile arch=linux-rocky8-x86_64
[+]  xojev5z          ^[email protected]%[email protected]~atomics~cuda~cxx~cxx_exceptions~gpfs~internal-hwloc~internal-libevent~internal-pmix~java~legacylaunchers~lustre~memchecker~openshmem~orterunprefix~romio+rsh~singularity~static+vt+wrapper-rpath build_system=autotools fabrics=none patches=073477a,60ce20b romio-filesystem=none schedulers=none arch=linux-rocky8-x86_64
[e]  6bfinv7              ^[email protected]%[email protected] build_system=autotools arch=linux-rocky8-x86_64
[+]  nql6rht              ^[email protected]%[email protected]~guile build_system=generic arch=linux-rocky8-x86_64
[+]  xvjgomg                  ^[email protected]%[email protected] build_system=generic arch=linux-rocky8-x86_64
[+]  fabsie6              ^[email protected]%[email protected]~cairo~cuda~gl~libudev+libxml2~netloc~nvml~oneapi-level-zero~opencl+pci~rocm build_system=autotools libs=shared,static arch=linux-rocky8-x86_64
[+]  ozoh3cd                  ^[email protected]%[email protected] build_system=autotools arch=linux-rocky8-x86_64
[+]  4siwbh2                      ^[email protected]%[email protected] build_system=autotools arch=linux-rocky8-x86_64
[+]  wfhyti5                  ^[email protected]%[email protected]+pic~python+shared build_system=autotools arch=linux-rocky8-x86_64
[+]  cebledz                      ^[email protected]%[email protected] build_system=autotools libs=shared,static arch=linux-rocky8-x86_64
[+]  dffqvjg                      ^[email protected]%[email protected]~pic build_system=autotools libs=shared,static arch=linux-rocky8-x86_64
[+]  hwpltg3              ^[email protected]%[email protected]+openssl build_system=autotools arch=linux-rocky8-x86_64
[+]  win2c3w                  ^[email protected]%[email protected]~docs+shared build_system=generic certs=mozilla arch=linux-rocky8-x86_64
[+]  kpg7le4                      ^ca-certificates-mozilla@2023-05-30%[email protected] build_system=generic arch=linux-rocky8-x86_64
[+]  f5p6lse              ^[email protected]%[email protected] build_system=autotools patches=4e1d78c,62fc8a8,ff37630 arch=linux-rocky8-x86_64
[+]  hmoiaig                  ^[email protected]%[email protected] build_system=autotools arch=linux-rocky8-x86_64
[+]  azbiry4                  ^[email protected]%[email protected] build_system=autotools arch=linux-rocky8-x86_64
[+]  cwstf4i                  ^[email protected]%[email protected] build_system=autotools arch=linux-rocky8-x86_64
[+]  zk4umu5                      ^[email protected]%[email protected] build_system=autotools patches=440b954 arch=linux-rocky8-x86_64
[+]  llftykr                  ^[email protected]%[email protected]+sigsegv build_system=autotools patches=9dc5fbd,bfdffa7 arch=linux-rocky8-x86_64
[+]  3zzhjea                      ^[email protected]%[email protected] build_system=autotools arch=linux-rocky8-x86_64
[+]  b2dal72                      ^[email protected]%[email protected] build_system=autotools arch=linux-rocky8-x86_64
[+]  ruajknv              ^[email protected]%[email protected]+gssapi build_system=autotools arch=linux-rocky8-x86_64
[+]  bpo3uds                  ^[email protected]%[email protected]+shared build_system=autotools arch=linux-rocky8-x86_64
[+]  2elq64s                      ^[email protected]%[email protected]~color build_system=autotools arch=linux-rocky8-x86_64
[+]  tujpbzt                      ^[email protected]%[email protected]+bzip2+curses+git~libunistring+libxml2+pic+shared+tar+xz build_system=autotools arch=linux-rocky8-x86_64
[+]  kvpossx                          ^[email protected]%[email protected]~debug~pic+shared build_system=generic arch=linux-rocky8-x86_64
[+]  tbq4kbc                          ^[email protected]%[email protected] build_system=autotools zip=pigz arch=linux-rocky8-x86_64
[+]  oy2avdp                              ^[email protected]%[email protected] build_system=makefile arch=linux-rocky8-x86_64
[+]  bjxklhc                              ^[email protected]%[email protected]+programs build_system=makefile compression=none libs=shared,static arch=linux-rocky8-x86_64
[+]  p7wxlwe                  ^[email protected]%[email protected] build_system=autotools arch=linux-rocky8-x86_64
[+]  wzpsuqq                  ^[email protected]%[email protected]~obsolete_api build_system=autotools patches=4885da3 arch=linux-rocky8-x86_64
[e]  3qsufzn              ^[email protected]%[email protected]+cpanm+opcode+open+shared+threads build_system=generic patches=8cf4302 arch=linux-rocky8-x86_64
[+]  magwyhz              ^[email protected]%[email protected]~docs+pmi_backwards_compatibility~python~restful build_system=autotools arch=linux-rocky8-x86_64
[+]  mhi2hph              ^[email protected]%[email protected]+compat+new_strategies+opt+pic+shared build_system=autotools arch=linux-rocky8-x86_64
[+]  j2zftyp          ^[email protected]%[email protected]+fortran~ipo~logging+mpi~pnetcdf~shared~timing build_system=cmake build_type=Release generator=make patches=55a6d7a arch=linux-rocky8-x86_64
[e]  7sn3ltj      ^[email protected]%[email protected] build_system=autotools arch=linux-rocky8-x86_64
[+]  3vliquc      ^[email protected]=2023.10.26%[email protected]~deterministic~ipo~optimisation_report build_system=cmake build_type=Release generator=make arch=linux-rocky8-x86_64
[+]  alths4d          ^[email protected]%[email protected]~doc+ncurses+ownlibs build_system=generic build_type=Release arch=linux-rocky8-x86_64
[+]  ne2hyzq              ^[email protected]%[email protected]~gssapi~ldap~libidn2~librtmp~libssh~libssh2+nghttp2 build_system=autotools libs=shared,static tls=openssl arch=linux-rocky8-x86_64
[+]  dsqyehb                  ^[email protected]%[email protected] build_system=autotools arch=linux-rocky8-x86_64
[+]  5n2mqbg              ^[email protected]%[email protected]~symlinks+termlib abi=none build_system=autotools patches=7a351bc arch=linux-rocky8-x86_64
[+]  gjylyhn          ^[email protected]%[email protected]~ipo build_system=cmake build_type=Release generator=make arch=linux-rocky8-x86_64
[+]  zujobr5          ^[email protected]%[email protected] build_system=autotools arch=linux-rocky8-x86_64
 -   oirvpqa      ^[email protected]_2024.08.14=master%[email protected]~access-gtracers~deterministic~optimisation_report+restart_repro build_system=makefile dev_path=/opt/environments/om2-develop/mom5 type=ACCESS-OM arch=linux-rocky8-x86_64

==> Updating view at /opt/environments/om2-develop/.spack-env/view

[harshula]

Alternate Solution

We could use spack config change, instead of editing the spack.yaml manually, once our spack.yaml files transition to using <package>@git.<gitref>=<spack-version> format.

This solution is requires the spack.yaml files to be updated. e.g. ACCESS-NRI/ACCESS-OM2#96