Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[1pt] PR: Fix return error status from pre-processing and remove CPU test #1075

Merged
merged 3 commits into from
Feb 16, 2024

Conversation

RobHanna-NOAA
Copy link
Contributor

@RobHanna-NOAA RobHanna-NOAA commented Feb 6, 2024

We recently added code to fim_pre_processing.sh that checks the CPU count. Earlier this test was being done in post-processing and was killing a pipeline that had already been running for a while.

Fix:

  • Removed the CPU test from pre-processing. This puts us back to it possibly failing in post-processing but we have to leave it for now.
  • Exit status codes (non 0) are now returned in pre-processing and post-processing when an error has occurred.

Tested that the a non zero return exit from pre-processing shuts down the AWS step functions.

Changes

  • fim_pre_processing.sh: added non zero exit codes when in error, plus removed CPU test
  • fim_post_processing.sh: added non zero exit codes when in error

Testing

  • Tested that a non-zero exit code is returned on arg errors and that AWS catches it and shuts down it's step function processing.
  • Tested a single huc fim_pipeline in local EC2.

Issuer Checklist (For developer use)

You may update this checklist before and/or after creating the PR. If you're unsure about any of them, please ask, we're here to help! These items are what we are going to look for before merging your code.

  • Informative and human-readable title, using the format: [_pt] PR: <description>
  • Links are provided if this PR resolves an issue, or depends on another other PR
  • If submitting a PR to the dev branch (the default branch), you have a descriptive Feature Branch name using the format: dev-<description-of-change> (e.g. dev-revise-levee-masking)
  • Changes are limited to a single goal (no scope creep)
  • The feature branch you're submitting as a PR is up to date (merged) with the latest dev branch
  • pre-commit hooks were run locally
  • Any change in functionality is tested
  • n/a - Passes all unit tests locally (inside interactive Docker container, at /foss_fim/, run: pytest unit_tests/)
  • n/a - New functions are documented (with a description, list of inputs, and expected output)
  • n/a - Placeholder code is flagged / future todos are captured in comments
  • CHANGELOG updated with template version number, e.g. 4.x.x.x
  • [ x Reviewers requested
  • Add yourself as an assignee in the PR as well as the FIM Technical Lead

Merge Checklist (For Technical Lead use only)

  • Update CHANGELOG with latest version number and merge date
  • Update the Citation.cff file to reflect the latest version number in the CHANGELOG
  • If applicable, update README with major alterations

@RobHanna-NOAA RobHanna-NOAA self-assigned this Feb 6, 2024
@RobHanna-NOAA RobHanna-NOAA added the bug Something isn't working label Feb 6, 2024
@RobHanna-NOAA RobHanna-NOAA linked an issue Feb 6, 2024 that may be closed by this pull request
@RobHanna-NOAA RobHanna-NOAA changed the title [1pt] Fix return error status from pre-processing and remove CPU test [1pt] PR: Fix return error status from pre-processing and remove CPU test Feb 6, 2024
@RobHanna-NOAA RobHanna-NOAA requested a review from mluck February 6, 2024 20:06
mluck
mluck previously approved these changes Feb 6, 2024
Copy link
Contributor

@mluck mluck left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems to work, although it's hard to tell if the exit 22 is actually being thrown.

@CarsonPruitt-NOAA CarsonPruitt-NOAA merged commit b7f8138 into dev Feb 16, 2024
1 check passed
@CarsonPruitt-NOAA CarsonPruitt-NOAA deleted the dev-fix-max-core-check branch February 16, 2024 17:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working FIM4
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[5pt] CPU test in pre-processing failed in AWS
3 participants