Updating to Dagster 1.9.7 results in DagsterInvalidDefinitionError on @asset_check definintions using additional_ins
#27003
Labels
Comments
|
Thanks for the report @zaneselvans . For now, we recommend pinning to 1.9.6, we're investigating the correct fix here. |
|
Sadly we also need the bump to using |
9 tasks
dpeng817
added a commit
that referenced
this issue
Jan 14, 2025
## Summary & Motivation Previously, `keys_by_input_name` did not contain the ins specified by additional_ins or additional_deps in the @asset_check decorator. This fixes to include those keys as part of the property. This was causing #27003 (comment) in some yet to be determined way. We've been able to confirm the fix, but have not yet pinpointed a straightforward repro. ## How I Tested These Changes - Fixes the problematic edge case. Add a bunch of new assertions to prevent regressions in the future. ## Changelog - Fixed a bug with asset checks where additional_deps/additional_ins were not being threaded through properly in certain cases, and would result in errors at job creation.
|
I'm also receiving duplicate assets problem when loading the dbt assets: |
|
@axellpadilla Is that happening when using |
Yes, tried with both 1.9.7 and 1.9.8 |
|
The merged commit to fix this issue will be in |
marijncv
pushed a commit
to marijncv/dagster
that referenced
this issue
Jan 21, 2025
) ## Summary & Motivation Previously, `keys_by_input_name` did not contain the ins specified by additional_ins or additional_deps in the @asset_check decorator. This fixes to include those keys as part of the property. This was causing dagster-io#27003 (comment) in some yet to be determined way. We've been able to confirm the fix, but have not yet pinpointed a straightforward repro. ## How I Tested These Changes - Fixes the problematic edge case. Add a bunch of new assertions to prevent regressions in the future. ## Changelog - Fixed a bug with asset checks where additional_deps/additional_ins were not being threaded through properly in certain cases, and would result in errors at job creation.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
What's the issue?
After updating our environment to use dagster 1.9.7, we are now getting a couple of instances of
DagsterInvalidDefinitionErrorwhen starting the dagster UI. This happens even when the only change is the dagster version bump. Previously our DAG had no such errors being flagged.Looking at the release notes associated with the 1.9.7 tag, the only change the seems like maybe it's related to this is the first item, the addition of
load_definitions_from_module. However, when attempting to read the documentation on that new feature, it doesn't appear on the page linked from the release notes, and I'm unable to find it searching the documentation, even though the dropdown at the top of the documentation says it pertains to dagster 1.9.7. I see that 1.9.7 isn't showing up in the changelog yet either.There seem to be a number of newly unresolvable inputs, and the Dagster UI startup bails as soon as it hits one, but there are several of them, and different ones get hit first on different attempts to start the UI. I have seen 3 different instances show up so far, and all of them are
asset_checks in which the supposedly unresolvable input is being provided usingadditional_in=AssetIn("some_asset_key_here")In our code base there is also a 4th instance of an
@asset_checktakingadditional_insbut I have not seen it be the one that causes the Dagster UI to fail yet.See code snippets and stack traces in additional information below.
What did you expect to happen?
I expected the Dagster UI to start up without any errors or warnings, as it does with v1.9.6.
How to reproduce?
I tried adding this simple example of the apparent pattern causing the failure to our project, but the other asset checks mentioned keep being the ones that cause the failure, so I don't know if this would cause the failure in isolation:
Dagster version
dagster, version 1.9.7
Deployment type
Local
Deployment details
Additional information
One of the problematic asset checks is being defined by an asset check factory, but it seems like the input it's complaining about (
tags_df) is indeed defined in another asset and then passed in explicitly usingadditional_in.Another problematic
@asset_checkis not produced by an asset factory, but the "unresolvable" input (aggs) is also being passed in usingadditional_in:Output from starting
dagster-webserverMessage from the maintainers
Impacted by this issue? Give it a 👍! We factor engagement into prioritization.
The text was updated successfully, but these errors were encountered: