Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[bug] Repeatability issue for adupa_winds_220 #313

Open
SamuelDegelia-NOAA opened this issue Mar 5, 2025 · 1 comment
Open

[bug] Repeatability issue for adupa_winds_220 #313

SamuelDegelia-NOAA opened this issue Mar 5, 2025 · 1 comment
Assignees

Comments

@SamuelDegelia-NOAA
Copy link
Contributor

SamuelDegelia-NOAA commented Mar 5, 2025

Current behavior (describe the bug)

While testing PR #298, I noticed that the ctests sometimes (not always fails) due to a TestReferenceFloatMismatchError on the cost function printout for adpupa_winds_220. The Jo stats are changing for some runs which leads to the ctest failure. Here are a sample of five runs for fv3-jedi EnVar that only assimilate adupa_winds_220:

Run 1: 
  0: Test Line: 'CostJo   : Nonlinear Jo(adpupa_winds_220) = 3.2593295738966270e+04, nobs = 48714, Jo/n = 6.6907451120758443e-01, err = 1.6247685156989053e+18'
Run 2: 
  0: Test Line: 'CostJo   : Nonlinear Jo(adpupa_winds_220) = 3.2593295738966270e+04, nobs = 48714, Jo/n = 6.6907451120758443e-01, err = 1.6247685156989053e+18'
Run 3: 
  0: Test Line: 'CostJo   : Nonlinear Jo(adpupa_winds_220) = 3.2592904892550017e+04, nobs = 48714, Jo/n = 6.6906648792031076e-01, err = 2.2988790520072753e+18'
Run 5: 
  0: Test Line: 'CostJo   : Nonlinear Jo(adpupa_winds_220) = 3.2596883201982317e+04, nobs = 48714, Jo/n = 6.6914815457532362e-01, err = 2.7916051027288340e+00'
Run 4: 
  0: Test Line: 'CostJo   : Nonlinear Jo(adpupa_winds_220) = 3.2596328854534120e+04, nobs = 48714, Jo/n = 6.6913677494219570e-01, err = 2.2999878982948780e+18'

I noticed the crash for the rrfs_fv3jedi_2024052700_Ens3Dvar ctest so I am not sure if this is a problem specific to fv3-jedi or if it also occurs for mpas-jedi.

Expected behavior

The output CostJo stats should not change for each run.

Suggested Solution

Will start by investigating if this is related to individual filters.

@SamuelDegelia-NOAA SamuelDegelia-NOAA self-assigned this Mar 5, 2025
@SamuelDegelia-NOAA
Copy link
Contributor Author

It looks like this results from ObsErrorFactorPressureCheck and only occurs for fv3-jedi. I ran EnVar ten times for mpas-jedi and the cost function stats were the same every time. I am not sure why the problem occurs but for now I will just comment out this filter for the fv3-jedi ctests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant