Merge pull request #730 from NREL/bpi2400

Sync tests
NREL · Apr 30, 2024 · 61d44fd · 61d44fd
2 parents af164e0 + 2c0f450
commit 61d44fd
Show file tree

Hide file tree

Showing 710 changed files with 12,011 additions and 11,522 deletions.
diff --git a/.github/workflows/config.yml b/.github/workflows/config.yml
@@ -170,12 +170,37 @@ jobs:
     runs-on: ubuntu-latest
     needs: [run-sample-files1-tests, run-sample-files2-tests, run-real-home-tests, run-other-tests]
     steps:
-      - name: Merge Artifacts
-        uses: actions/upload-artifact/merge@v4
+      - uses: actions/checkout@v4
+        with:
+          ref: ${{ github.head_ref }}
+
+      - name: Download all results
+        uses: actions/download-artifact@v4
         with:
-          name: results
+          path: test_results
           pattern: test_results_*
-          delete-merged: true
+          merge-multiple: true
+
+      - name: Delete artifacts
+        uses: geekyeggo/delete-artifact@v5
+        with:
+          name: test_results_*
+
+      - name: Merge results
+        run: |
+          mkdir workflow/tests/test_results
+          cp -r test_results/*.csv workflow/tests/test_results
+          cd workflow/tests
+          pip install pandas
+          python ../../hpxml-measures/workflow/tests/merge.py test_results/sample_files1.csv test_results/sample_files2.csv test_results/sample_files.csv
+          rm test_results/sample_files1.csv
+          rm test_results/sample_files2.csv
+
+      - name: Store results
+        uses: actions/upload-artifact@v4
+        with:
+          path: workflow/tests/test_results
+          name: test_results
 
   compare-results:
     if: github.event_name == 'pull_request'
@@ -205,14 +230,14 @@ jobs:
       - name: Download feature results
         uses: actions/download-artifact@v4
         with:
-          path: results
-          name: results
+          path: test_results
+          name: test_results
 
       - name: Compare results
         run: |
           cp -r base_results/*.csv workflow/tests/base_results
-          mkdir workflow/tests/results
-          cp -r results/*.csv workflow/tests/results
+          mkdir workflow/tests/test_results
+          cp -r test_results/*.csv workflow/tests/test_results
           
           pip install numpy
           pip install pandas
@@ -239,15 +264,15 @@ jobs:
       - name: Download feature results
         uses: actions/download-artifact@v4
         with:
-          path: results
-          name: results
+          path: test_results
+          name: test_results
 
       - name: Commit latest results
         shell: bash        
         run: |
           branch_name="${{ github.head_ref }}"
           git pull origin $branch_name
-          cp -r results/*.csv workflow/tests/base_results
+          cp -r test_results/*.csv workflow/tests/base_results
           git add workflow/tests/base_results
           git status
           if [[ $(git diff --cached --exit-code) ]]; then

diff --git a/Changelog.md b/Changelog.md
@@ -9,17 +9,19 @@ __New Features__
   - The `ElectricAuxiliaryEnergy` input for boilers is no longer used.
 - **Breaking change**: ERI_Results.csv and ERI_Worksheet.csv combined into a single ERI_Results.csv that better reflects the current ERI calculation components; additional fields (e.g., PEfrac) added and a few renamed/removed.
 - **Breaking change**: Each `VentilationFan` must have one (and only one) `UsedFor...` element set to true.
+- HERS software tests:
+  - HERS Hot Water test HPXMLs have been updated to be direct EnergyPlus simulations (like the HERS HVAC & DSE tests already were); they are no longer run through ERI simulations.
+  - HERS HVAC DSE tests now use duct effective R-values instead of nominal insulation R-values to demonstrate that they pass RESNET acceptance criteria.
 - Allows optional `Ducts/DuctShape` and `Ducts/DuctFractionRectangular` inputs, which affect duct effective R-value used for modeling.
 - Allows `AverageCeilingHeight` to be optionally provided for infiltration calculations.
-- Ground source heat pump model enhancements.
 - Allows `Roof/RadiantBarrier` to be omitted; defaults to false.
-- Adds optional `Slab/extension/GapInsulationRValue` input for cases where a slab has horizontal (under slab) insulation.
-- Adds more error-checking for inappropriate inputs (e.g., HVAC SHR=0 or clothes washer IMEF=0).
 - Allows `FractionDuctArea` as alternative to `DuctSurfaceArea`
+- Allows alternative `LabelEnergyUse` (W) input for ceiling fans.
+- Allows `Slab/extension/GapInsulationRValue` input for cases where a slab has horizontal (under slab) insulation.
+- Ground source heat pump model enhancements.
 - Now defaults to -20F for `CompressorLockoutTemperature` for variable-speed air-to-air heat pumps.
-- Clarifies that Capacity=-1 can be used to autosize HVAC equipment for research purposes or to run tests.
-  - It should *not* be used for a real home; a warning will be issued when it's used.
-- Allow alternative label energy use (W) input for ceiling fans.
+- Adds more error-checking for inappropriate inputs (e.g., HVAC SHR=0 or clothes washer IMEF=0).
+- Clarifies that HVAC `Capacity=-1` can be used to autosize HVAC equipment for research purposes or to run tests. It should *not* be used for a real home, and a warning will be issued when it's used.
 
 __Bugfixes__
 - Fixes incorrect Reference Home mechanical ventilation flowrate for attached units (when Aext is not 1).

diff --git a/docs/source/testing_framework.rst b/docs/source/testing_framework.rst
@@ -30,17 +30,16 @@ Or individual tests can be run by specifying the name of the test. A couple exam
 | ``openstudio es_zerh_test.rb --name=test_epa``
 | 
 
-Test results in CSV format are created at workflow/tests/test_results. 
-For many RESNET tests, the Excel spreadsheet test criteria are also implemented in code to automate the process of checking for test failures.
-All simulation/HPXML/etc. files generated from running the tests can be found inside the workflow/tests/test_files directory.
+Test results in CSV format are created at ``workflow/tests/test_results`` and can be used to populate RESNET Excel spreadsheet forms. 
+RESNET acceptance criteria are also implemented as part of the tests to check for test failures.
 
-At the completion of the test, there will also be output that denotes the number of failures/errors like so:
+At the completion of the test, there will be output that denotes the number of failures/errors like so:
 
 ``Finished in 36.067116s, 0.0277 runs/s, 0.9704 assertions/s.``
 ``1 runs, 35 assertions, 0 failures, 0 errors, 0 skips``
 
 Software developers may find it convenient to export HPXML files with the same name as the test files included in the repository.
-This allows issuing the same commands above to generate test results.
+This allows issuing the commands above to generate test results.
 
 Official Test Results
 ---------------------

diff --git a/hpxml-measures/.github/workflows/config.yml b/hpxml-measures/.github/workflows/config.yml
@@ -46,7 +46,7 @@ jobs:
       - name: Store results
         uses: actions/upload-artifact@v4
         with:
-          path: workflow/tests/results
+          path: workflow/tests/test_results
           name: results-unit-tests
 
       - name: Build documentation
@@ -85,7 +85,7 @@ jobs:
       - name: Store results
         uses: actions/upload-artifact@v4
         with:
-          path: workflow/tests/results
+          path: workflow/tests/test_results
           name: results-workflow1-tests
 
   run-workflow2-tests:
@@ -108,7 +108,7 @@ jobs:
       - name: Store results
         uses: actions/upload-artifact@v4
         with:
-          path: workflow/tests/results
+          path: workflow/tests/test_results
           name: results-workflow2-tests
 
   run-windows-tests:
@@ -130,12 +130,35 @@ jobs:
     runs-on: ubuntu-latest
     needs: [run-workflow1-tests, run-workflow2-tests, run-unit-tests]
     steps:
-      - name: Merge Artifacts
-        uses: actions/upload-artifact/merge@v4
+      - uses: actions/checkout@v4
         with:
-          name: results
+          ref: ${{ github.head_ref }}
+
+      - name: Download all results
+        uses: actions/download-artifact@v4
+        with:
+          path: test_results
           pattern: results-*
-          delete-merged: true
+          merge-multiple: true
+
+      - name: Delete artifacts
+        uses: geekyeggo/delete-artifact@v5
+        with:
+          name: results-*
+
+      - name: Merge results
+        run: |
+          mkdir workflow/tests/test_results
+          cp -r test_results/results* workflow/tests/test_results
+          cd workflow/tests
+          pip install pandas
+          for csv in test_results/*1_*.csv; do csv2="${csv/1_/2_}"; csv_out="${csv/1_/_}"; python merge.py $csv $csv2 $csv_out; rm $csv; rm $csv2; done
+
+      - name: Store results
+        uses: actions/upload-artifact@v4
+        with:
+          path: workflow/tests/test_results
+          name: test_results
 
   compare-results:
     if: github.event_name == 'pull_request'
@@ -165,14 +188,14 @@ jobs:
       - name: Download feature results
         uses: actions/download-artifact@v4
         with:
-          path: results
-          name: results
+          path: test_results
+          name: test_results
 
       - name: Compare results
         run: |
           cp -r base_results/results*.csv workflow/tests/base_results
-          mkdir workflow/tests/results
-          cp -r results/results* workflow/tests/results
+          mkdir workflow/tests/test_results
+          cp -r test_results/results* workflow/tests/test_results
           
           pip install numpy
           pip install pandas
@@ -199,15 +222,15 @@ jobs:
       - name: Download feature results
         uses: actions/download-artifact@v4
         with:
-          path: results
-          name: results
+          path: test_results
+          name: test_results
 
       - name: Commit latest results
         shell: bash        
         run: |
           branch_name="${{ github.head_ref }}"
           git pull origin $branch_name
-          cp -r results/results* workflow/tests/base_results
+          cp -r test_results/results* workflow/tests/base_results
           git add workflow/tests/base_results
           git status
           if [[ $(git diff --cached --exit-code) ]]; then

diff --git a/hpxml-measures/.gitignore b/hpxml-measures/.gitignore
@@ -1005,7 +1005,7 @@
 /workflow/reports
 /workflow/out.osw
 /workflow/tests/results
-/workflow/tests/run
+/workflow/tests/run*
 /workflow/tests/test*
 /workflow/sample_files/results
 /workflow/sample_files/run*/