Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automate building external libraries #694

Merged
merged 32 commits into from
Jan 25, 2024
Merged

Automate building external libraries #694

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
cde8857
cmake: automate building extern libraries; see extended commit msg
program-- Dec 28, 2023
30b5c8c
Be more exact in disabling SLoTH testing
PhilMiller Dec 28, 2023
80249a6
Try to let CMake build PET automatically
PhilMiller Dec 28, 2023
225e303
Declare dependence of PET test on PET external module
PhilMiller Dec 28, 2023
5b4f2ff
Bugfix: use correctly prefixed variable name
PhilMiller Dec 28, 2023
cde3b46
cmake: update status info; fix ngen_add_tests depends; set bmi_c ON f…
program-- Dec 28, 2023
ed7c267
cmake: fix dependency typo in ngen_add_test
program-- Dec 28, 2023
1555ca8
cmake: refactor iso_c_bmi/noah-owp-modular listfiles; add add_fortran…
program-- Dec 29, 2023
3275f63
cmake: refactor test_bmi_fortran listfile
program-- Dec 29, 2023
4f3f06e
cmake: replace add_fortran_library so projects can be built independe…
program-- Dec 29, 2023
1a2860b
gh: rm building submodules for fortran BMI tests
program-- Dec 29, 2023
beaa72c
cmake: fix target link, bmi_fortran_iso_c -> iso_c_bmi
program-- Dec 29, 2023
cf97f6a
cmake: change scope of compile options to PRIVATE to prevent non-port…
program-- Dec 29, 2023
90d218c
gh: attempt to specify fortran compiler for test_bmi_fortran job
program-- Dec 29, 2023
fb94360
gh: add fortran compiler setup step for macOS runners when bmi_fortra…
program-- Dec 29, 2023
02ba321
cmake: add correct requirements/dependencies for test_bmi_multi
program-- Dec 29, 2023
a148143
gh: fix placement of fortran env step; cmake: add missing ffree-line-…
program-- Dec 29, 2023
cdca20c
gh: remove submod build steps for integration tests
program-- Dec 29, 2023
19f6e5e
gh: refactor docker workflow
program-- Dec 29, 2023
0279bfc
docker: revert pybind/gtest submodule updates; cmake: ensure sloth ha…
program-- Dec 29, 2023
9d7b831
docker: include git files in build context
program-- Jan 2, 2024
6390adb
cmake: rm find bmi_fortran_iso_c module
program-- Jan 2, 2024
7bbb846
cmake: update submodules on configure
program-- Jan 2, 2024
f5e3860
cmake: update extern bmi-cxx
program-- Jan 2, 2024
98320e7
cmake: add recursive flag to git submodules function
program-- Jan 2, 2024
94481f3
cmake: finer options for building models; default visibility for impo…
program-- Jan 3, 2024
a538d33
cmake: revert visibility modification to just sloth
program-- Jan 3, 2024
d211c61
cmake: add ngen_with_extern_all option
program-- Jan 5, 2024
3827696
cmake: rm fully-deprecated option; ensure all support
program-- Jan 11, 2024
341c03b
gh: rm submodule build steps
program-- Jan 11, 2024
3576b76
docker: update boost URL
program-- Jan 11, 2024
0c378c0
docker: bump boost version; gh: bump checkout to v4
program-- Jan 25, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .github/actions/ngen-build/action.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
name: 'Checkout and build tests'

description: 'This action checks out the commit, sets up Ngen environemnt.'
author: 'hellkite500'
inputs:
Expand Down Expand Up @@ -165,6 +166,11 @@ runs:
deactivate
shell: bash

- name: Setup Fortran Compiler
if: ${{ inputs.bmi_fortran == 'ON' && runner.os == 'macOS' }}
run: echo "FC=gfortran-13" >> $GITHUB_ENV
shell: bash

- name: Cmake Initialization
id: cmake_init
run: |
Expand Down
14 changes: 8 additions & 6 deletions .github/workflows/Example_model_run.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,13 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout the code
uses: actions/checkout@v3
uses: actions/checkout@v4

- name: Build the NGEN image
run: docker build . --file ./docker/CENTOS_NGEN_RUN.dockerfile --tag localbuild/ngen:latest
run: docker build --file ./docker/ngen.dockerfile --tag localbuild/ngen:latest .

- name: Run the NGEN model on example data
run: docker run --rm -i localbuild/ngen:latest
#Internal command has hardcoded paths to repo-local data
#When running command manually from container use below:
#./ngen data/catchment_data.geojson "" data/nexus_data.geojson "" data/refactored_example_realization_config.json
run: |
docker run --rm -i --mount type=bind,source="$(pwd)"/data,target=/ngen_build/data,readonly \
localbuild/ngen:latest \
./ngen data/catchment_data.geojson '' data/nexus_data.geojson '' data/example_realization_config.json
37 changes: 1 addition & 36 deletions .github/workflows/module_integration.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,46 +31,11 @@ jobs:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout@v3

- name: Build Surfacebmi
id: submod_build_1
uses: ./.github/actions/ngen-submod-build
with:
mod-dir: " extern/noah-owp-modular/"
targets: "surfacebmi"

- name: Build ISO C Fortran BMI
id: submod_build_2
uses: ./.github/actions/ngen-submod-build
with:
mod-dir: "extern/iso_c_fortran_bmi/"

- name: Build CFE
id: submod_build_3
uses: ./.github/actions/ngen-submod-build
with:
mod-dir: "extern/cfe/"
targets: "cfebmi"

- name: Build PET
id: submod_build_5
uses: ./.github/actions/ngen-submod-build
with:
mod-dir: "extern/evapotranspiration/evapotranspiration"
targets: "petbmi"

- name: Build SLoTH
id: submod_build_6
uses: ./.github/actions/ngen-submod-build
with:
mod-dir: "extern/sloth/"
targets: "slothmodel"

- name: Build Ngen
- name: Build NGen
uses: ./.github/actions/ngen-build
with:
targets: "ngen"
build-cores: ${{ env.LINUX_NUM_PROC_CORES }}
#is this required for this test?
bmi_c: 'ON'
bmi_fortran: 'ON'
timeout-minutes: 15
Expand Down
98 changes: 1 addition & 97 deletions .github/workflows/test_and_validate.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,17 +68,11 @@ jobs:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout@v3

- name: Build PET Submodule
id: submod_build_5
uses: ./.github/actions/ngen-submod-build
with:
mod-dir: "extern/evapotranspiration/evapotranspiration"
targets: "petbmi"

- name: Build PET Tests
uses: ./.github/actions/ngen-build
with:
targets: "compare_pet"
bmi_c: on
build-cores: ${{ env.LINUX_NUM_PROC_CORES }}

- name: Run Tests
Expand Down Expand Up @@ -136,13 +130,6 @@ jobs:
with:
mod-dir: "extern/bmi-cxx/"

- name: Build Submodules
id: submod_build
uses: ./.github/actions/ngen-submod-build
with:
mod-dir: "extern/test_bmi_cpp/"
targets: "testbmicppmodel"

- name: Build Unit Tests
uses: ./.github/actions/ngen-build
with:
Expand All @@ -159,11 +146,6 @@ jobs:
- name: Clean Up Unit Test Build
uses: ./.github/actions/clean-build

- name: Clean Up Submodule Build
uses: ./.github/actions/clean-build
with:
build-dir: ${{ steps.submod_build.outputs.build-dir }}

# Run BMI C tests in linux environment, and separating to isolate
test_bmi_c:
# The type of runner that the job will run on
Expand All @@ -178,13 +160,6 @@ jobs:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout@v3

- name: Build Submodules
id: submod_build
uses: ./.github/actions/ngen-submod-build
with:
mod-dir: "extern/test_bmi_c"
targets: "testbmicmodel"

- name: Build Unit Tests
uses: ./.github/actions/ngen-build
with:
Expand All @@ -204,11 +179,6 @@ jobs:
- name: Clean Up Unit Test Build
uses: ./.github/actions/clean-build

- name: Clean Up Submodule Build
uses: ./.github/actions/clean-build
with:
build-dir: ${{ steps.submod_build.outputs.build-dir }}

# As with the BMI C ubuntu job, separate Fortran in linux tests to keep setups clean and cause of failures clear.
test_bmi_fortran:
# The type of runner that the job will run on
Expand All @@ -223,18 +193,6 @@ jobs:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout@v3

- name: Build Submodules
id: submod_build_1
uses: ./.github/actions/ngen-submod-build
with:
mod-dir: "extern/test_bmi_fortran/"

- name: Build Submodules
id: submod_build_2
uses: ./.github/actions/ngen-submod-build
with:
mod-dir: "extern/iso_c_fortran_bmi/"

- name: Build Unit Tests
uses: ./.github/actions/ngen-build
with:
Expand All @@ -254,15 +212,6 @@ jobs:
- name: Clean Up Unit Test Build
uses: ./.github/actions/clean-build

- name: Clean Up Submodule Build
uses: ./.github/actions/clean-build
with:
build-dir: ${{ steps.submod_build_1.outputs.build-dir }}

- uses: ./.github/actions/clean-build
with:
build-dir: ${{ steps.submod_build_2.outputs.build-dir }}

# As with the BMI C ubuntu job, separate Python in linux tests to keep setups clean and cause of failures clear.
test_bmi_python:
# The type of runner that the job will run on
Expand Down Expand Up @@ -312,37 +261,6 @@ jobs:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout@v3

- name: Build Submodules
id: submod_build_1
uses: ./.github/actions/ngen-submod-build
with:
mod-dir: "extern/test_bmi_fortran/"

- name: Build Submodules
id: submod_build_2
uses: ./.github/actions/ngen-submod-build
with:
mod-dir: "extern/iso_c_fortran_bmi/"

- name: Build Submodules
id: submod_build_3
uses: ./.github/actions/ngen-submod-build
with:
mod-dir: "extern/test_bmi_c/"
targets: "testbmicmodel"

#make sure cxx bmi is initialized/ready
- uses: ./.github/actions/ngen-submod-build
with:
mod-dir: "extern/bmi-cxx/"

- name: Build Submodules
id: submod_build_4
uses: ./.github/actions/ngen-submod-build
with:
mod-dir: "extern/test_bmi_cpp/"
targets: "testbmicppmodel"

- name: Build Unit Tests
uses: ./.github/actions/ngen-build
with:
Expand All @@ -362,20 +280,6 @@ jobs:
- name: Clean Up Unit Test Build
uses: ./.github/actions/clean-build

- name: Clean Up Submodule Build
uses: ./.github/actions/clean-build
with:
build-dir: ${{ steps.submod_build_1.outputs.build-dir }}
- uses: ./.github/actions/clean-build
with:
build-dir: ${{ steps.submod_build_2.outputs.build-dir }}
- uses: ./.github/actions/clean-build
with:
build-dir: ${{ steps.submod_build_3.outputs.build-dir }}
- uses: ./.github/actions/clean-build
with:
build-dir: ${{ steps.submod_build_4.outputs.build-dir }}


# TODO: fails due to compilation error, at least in large part due to use of POSIX functions not supported on Windows.
# TODO: Need to determine whether Windows support (in particular, development environment support) is necessary.
Expand Down
Loading
Loading