Merge branch 'develop' into download

.github/workflows/test.yml

@@ -25,8 +25,8 @@ jobs:
           ignore_words_list: atleast,ninjs,simpy,proovread,namd,precice,crate,ake
           # filter out
           #   docs/js/asciinema-player-2.6.1.js as it is not markdown
-          #   version-specific/supported-software.md as the software descriptions have spelling errors
-          skip: './docs/js/asciinema-player-2.6.1.js,./docs/version-specific/supported-software.md,./docs/release-notes.md'
+          #   version-specific/supported-software/* as the software descriptions have spelling errors
+          skip: './docs/js/asciinema-player-2.6.1.js,./docs/version-specific/supported-software/*,./docs/version-specific/supported-software/*/*.md,./docs/release-notes.md'
 
       - name: check internal links
         run: python ./.github/workflows/link_check.py

docs/common-toolchains.md

@@ -51,8 +51,8 @@ graph LR
   A --> |binutils| C[intel-compilers];
   B --> |OpenMPI| E[gompi];
   C --> |impi| F[iimpi];
-  B --> |FlexiBLAS + FFTW + ScaLAPACK| D[gfbf];
-  D --> |OpenMPI| G[foss];
+  B --> |"FlexiBLAS (incl. LAPACK) + FFTW"| D[gfbf];
+  D --> |OpenMPI + ScaLAPACK| G[foss];
   E --> |FlexiBLAS + FFTW + ScaLAPACK| G[foss];
   F --> |imkl| Z[intel];
   C --> |imkl| H[iimkl];
@@ -61,9 +61,8 @@ graph LR
 
 Note: following notes apply for the generations listed and those older than it:
 
-- `2022a` - `iimkl` not present yet
-- `2021b` - `gfbf` not present yet
-- `2020b` - `foss` uses OpenBLAS instead of FlexiBLAS, `iccifort` is used instead of `intel-compilers`
+- <= `2021b` - `gfbf` not present yet
+- <= `2020b` - `foss` uses OpenBLAS instead of FlexiBLAS, `iccifort` is used instead of `intel-compilers`
 
 
 Keep in mind that when creating an Easyconfig, you need to look at what toolchain "level" (e.g. `foss` vs `GCC`) your
@@ -180,6 +179,7 @@ most recent revision of the common toolchains at that time.
 | `2022b` | Dec '22  | 2.39       | 12.2.0 | 4.1.4      | 3.2.1       | 0.3.21     | (incl. with FlexiBLAS) | 2.2.0       | 3.3.10 |
 | `2023a` | Jun '23  | 2.40       | 12.3.0 | 4.1.5      | 3.3.1       | 0.3.23     | (incl. with FlexiBLAS) | 2.2.0       | 3.3.10 |
 | `2023b` | Dec '23  | 2.40       | 13.2.0 | 4.1.6      | 3.3.1       | 0.3.24     | (incl. with FlexiBLAS) | 2.2.0       | 3.3.10 |
+| `2024a` | Aug '24  | 2.42       | 13.3.0 | 5.0.3      | 3.4.4       | 0.3.27     | (incl. with FlexiBLAS) | 2.2.0       | 3.3.10 |
 
 ### Component versions in `intel` toolchain {: #common_toolchains_overview_intel }
 
@@ -191,6 +191,7 @@ most recent revision of the common toolchains at that time.
 | `2022b` | Dec '22  | 2.39       | 12.2.0 | 2022.2.1          | 2021.7.1    | 2022.2.1    |
 | `2023a` | Jun '23  | 2.40       | 12.3.0 | 2023.1.0          | 2021.9.1    | 2023.1.0    |
 | `2023b` | Dec '23  | 2.40       | 13.2.0 | 2023.2.1          | 2021.10.1   | 2023.2.0    |
+| `2024a` | Aug '24  | 2.42       | 13.3.0 | 2024.2.0          | 2021.13.0   | 2024.2.0    |
 
 ## Overview of common toolchains (deprecated versions) {: #common_toolchains_overview_deprecated }
 

docs/easybuild-v5/changes-in-default-configuration.md

@@ -1,17 +1,17 @@
 # Changes in default configuration in EasyBuild v5.0
 
-*(for a full overview of changes in EasyBuild v5.0, see [here](overview-of-changes.md))*
+*(for a full overview of changes in EasyBuild v5.0, see [here](index.md))*
 
-Various changes in default configuration included in EasyBuild v5.0, including:
+The default value for several EasyBuild configuration settings has been changed in EasyBuild v5.0:
 
-* [Enable RPATH linking by default][rpath]
-* [Enable `--trace` by default][trace]
+- [RPATH linking is enabled by default (`--rpath`)](rpath)
+- [Trace output is enabled by default (`--trace`)](trace)
 
 ---
 
-## Enable RPATH linking by default {: #rpath }
+## RPATH linking is enabled by default (`--rpath`) {: #rpath }
 
-[RPATH linking][rpath_support] is enabled by default in EasyBuild v5.0 (see [easybuild-framework PR #4448](https://github.com/easybuilders/easybuild-framework/pull/4448)).
+[RPATH linking][rpath_support] is enabled by default in EasyBuild v5.0.
 
 The benefits for enabling RPATH are explained in [Why RPATH?][rpath_support_why].
 
@@ -20,31 +20,34 @@ will continue to be appended by the environment module files EasyBuild generates
 unless it is configured to filter these variables (via `--filter-env-vars`,
 see also [Relation to `$LD_LIBRARY_PATH`][rpath_support_LD_LIBRARY_PATH]).
 
+The RPATH part of the EasyBuild sanity check has been relaxed (by default,
+to allow for installing software that uses RPATH linking on top of existing software installations that do not use RPATH.
+The RPATH sanity check can be made strict again via the `strict-rpath-sanity-check` EasyBuild configuration setting.
+
 To disable RPATH linking, either:
 
-* Use the `--disable-rpath` command line option;
-* Set the `$EASYBUILD_DISABLE_RPATH` environment variable;
-* Disable RPATH linking in an EasyBuild [configuration file](../configuration.md#configuration_file):
+- Use the `--disable-rpath` command line option;
+- Set the `$EASYBUILD_DISABLE_RPATH` environment variable;
+- Disable RPATH linking in an EasyBuild [configuration file](../configuration.md#configuration_file):
 
     ``` ini
     [override]
     rpath=0
     ```
 
-
 ---
 
-## Enable `--trace` by default {: #trace }
+## Trace output is enabled by default (`--trace`) {: #trace }
 
-The [`--trace` option](../tracing-progress.md) is enabled by default (see [easybuild-framework PR #4250](https://github.com/easybuilders/easybuild-framework/pull/4250)).
+The [`--trace` option](../tracing-progress.md) is enabled by default.
 
 This makes the output produced by the `eb` command more informative, by providing more information about what's going on in the background.
 
 To disable trace output, either:
 
-* Use the `--disable-trace` command line option;
-* Set the `$EASYBUILD_DISABLE_TRACE` environment variable;
-* Disable trace mode in a [configuration file](../configuration.md#configuration_file):
+- Use the `--disable-trace` command line option;
+- Set the `$EASYBUILD_DISABLE_TRACE` environment variable;
+- Disable trace mode in a [configuration file](../configuration.md#configuration_file):
 
     ``` ini
     [override]

docs/easybuild-v5/deprecated-functionality.md

@@ -1,19 +1,20 @@
 # Deprecated functionality in EasyBuild v5.0
 
-*(for a full overview of changes in EasyBuild v5.0, see [here](overview-of-changes.md))*
+*(for a full overview of changes in EasyBuild v5.0, see [here](index.md))*
 
 Some functionality is being deprecated in EasyBuild v5.0, and will no longer be supported in EasyBuild v6.0:
 
-* [`run_cmd` and `run_cmd_qa` functions][run_cmd]
-* [`easybuild.tools.py2vs3` module][py2vs3]
+- [`run_cmd` and `run_cmd_qa` functions][run_cmd]
+- [`easybuild.tools.py2vs3` module][py2vs3]
 
 If you trigger any deprecated functionality when using EasyBuild v5.0, a warning message will be printed.
 
 ---
 
 ## `run_cmd` and `run_cmd_qa` functions {: #run_cmd }
 
-The `run_cmd` and `run_cmd_qa` functions will be deprecated.
+The `run_cmd` and `run_cmd_qa` functions have been deprecated,
+and are scheduled to be removed in EasyBuild v6.0.
 
 You should migrate to the new [`run_shell_cmd`](run_shell_cmd.md) function instead.
 
@@ -22,8 +23,6 @@ You should migrate to the new [`run_shell_cmd`](run_shell_cmd.md) function inste
 
 ## `easybuild.tools.py2vs3` module {: #py2vs3 }
 
-[easybuild-framework PR #4229](https://github.com/easybuilders/easybuild-framework/pull/4229)
-
 The following table lists the changes required to replace imports from the the `py2vs3` module.
 
 | `from easybuild.tools.py2vs3 import ...` | Replacement |

docs/easybuild-v5/enhancements.md

@@ -1,10 +1,12 @@
 # Enhancements in EasyBuild v5.0
 
-*(for a full overview of changes in EasyBuild v5.0, see [here](overview-of-changes.md))*
+*(for a full overview of changes in EasyBuild v5.0, see [here](index.md))*
 
 Various significant enhancements are included in EasyBuild v5.0, including:
 
-* [`run_shell_cmd` function][run_shell_cmd]
+- [`run_shell_cmd` function][run_shell_cmd]
+- [Granular exit codes][granular_exit_codes]
+- [Reproducible tarballs][reproducible_tarballs]
 * [Require `download_instructions` to be specified][require_download_instructions]
 
 ---
@@ -15,6 +17,37 @@ See dedicated page on the new [`run_shell_cmd` function](run_shell_cmd.md).
 
 ---
 
+## Granular exit codes { : #granular_exit_codes }
+
+EasyBuild v5 now uses a range of ~50 exit codes instead of just 0 for normal
+termination and 1 for unexpected termination. Each non-zero exit code
+correlates to the specific type of error or failure that caused the
+termination of the program. For instance, a missing easyconfig or a failed
+checksum check. The full list of exit codes is defined in the class
+[easybuild.tools.build_log.EasyBuildExit](https://github.com/easybuilders/easybuild-framework/blob/main/easybuild/tools/build_log.py#L74).
+
+EasyBuild will always return its own exit codes on termination. Other exit
+codes from external processes executed through `run_shell_cmd` or HTTP response
+status codes are reported in the corresponding logs.
+
+---
+
+## Reproducible tarballs { : #reproducible_tarballs}
+
+EasyBuild can now generate reproducible tarballs of sources cloned from Git
+repositories. This means that those sources using the `git_config` option will
+now have consistent contents across different systems and across time, allowing
+to reliably validate them with checksums. EasyBuild follows the
+[archival guidelines from reproducible-builds.org](https://reproducible-builds.org/docs/archives/) 
+to generate those reproducible tarballs.
+
+This new feature does not apply to sources cloned with `keep_git_dir` enabled.
+Including the `.git` folder in the sources is inherently time-dependent as it
+contains information about the clone action itself, which hinders the creation
+of a reproducible tarball.
+
+---
+
 ## Require `download_instructions` to be specified {: require_download_instructions }
 
 [`download_instructions`][download_instructions] is used in easyconfigs to specify instructions, or information, on

docs/easybuild-v5/index.md

@@ -1,11 +1,91 @@
 # EasyBuild v5.0
 
-- [Overview of changes](overview-of-changes.md)
-    - [Backwards-incompatible changes](backwards-incompatible-changes.md)
-    - [Changes in default configuration](changes-in-default-configuration.md)
-    - [Enhancements](enhancements.md)
-        - [`run_shell_cmd` function](run_shell_cmd.md)
-    - [Deprecated functionality](deprecated-functionality.md)
-    - [Policies](policies.md)
-- [GitHub Project board](https://github.com/orgs/easybuilders/projects/18)
+A high-level overview of changes in EasyBuild version 5.0 is listed below.
+
+Click on a particular item for more information.
+
+!!! warning "EasyBuild version 5.0.0 is not released yet"
+
+    EasyBuild v5.0.0 is still under active development via the `5.0.x` branches in the EasyBuild GitHub repositories.
+
+    Shortly before the release of EasyBuild v5.0.0, the `5.0.x` branches will be collapsed in the corresponding
+    `develop` branches, and the `5.0.x` branches will then only be used as a staging area for additional EasyBuild
+    v5.0.x releases.
+
+    The overview below is known to be incomplete, and will be gradually completed as we approach
+    the release of EasyBuild v5.0.0.
+
+---
+
+## Breaking changes {: #breaking-changes }
+
+EasyBuild v5.0 includes a number of backwards-incompatible changes:
+
+- [Python 3.6+ is required to run EasyBuild v5.0.0](python36-required.md)
+
+See also the [overview of removed functionality][removed-functionality] below.
+
+
+---
+
+## Changes in default configuration
+
+The default value for several EasyBuild configuration settings has been changed in EasyBuild v5.0:
+
+- [RPATH linking is enabled by default (`--rpath`)](changes-in-default-configuration.md#rpath)
+- [Trace output is enabled by default (`--trace`)](changes-in-default-configuration.md#trace)
+
+
+---
+
+## Enhancements
+
+Various significant enhancements are included in EasyBuild v5.0, including:
+
+- [`run_shell_cmd` function](run_shell_cmd.md)
+- [Granular exit codes](enhancements.md#granular_exit_codes)
+- [Reproducible tarballs](enhancements.md#reproducible_tarballs)
+
+
+---
+
+## Removed functionality {: #removed-functionality }
+
+Functionality that was deprecated a while ago has been removed:
+
+- [EasyBuild bootstrap script is removed](removed-functionality.md#bootstrap_script) 
+- [Experimental support for `.yeb` easyconfig format is removed](removed-functionality.md#yeb)
+
+
+---
+
+## Deprecated functionality {: #deprecated-functionality }
+
+Some functionality is being deprecated in EasyBuild v5.0, and is scheduled to be removed in EasyBuild v6.0:
+
+- [`run_cmd` and `run_cmd_qa` functions](deprecated-functionality.md#run_cmd)
+- [`easybuild.tools.py2vs3` module](deprecated-functionality.md#py2vs3)
+
+
+---
+
+## Other changes
+
+*(coming soon)*
+
+
+---
+
+## FAQ
+
+*(coming soon)*
+
+
+---
+
+## Other links
+
+- [Beta releases and release candidates](release-candidates.md)
+- [GitHub Project board for EasyBuild v5.0](https://github.com/orgs/easybuilders/projects/18)
 - [Talk on EasyBuild 5.0 at EasyBuild User Meeting 2023](https://easybuild.io/eum23/#easybuild5)
+- [Talk on EasyBuild 5.0 at EasyBuild User Meeting 2024](https://easybuild.io/eum24/#eb5)