Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable filtering by year for EIA 861 and FERC 714 ETLs #2628

Open
1 of 2 tasks
e-belfer opened this issue Jun 5, 2023 · 0 comments · Fixed by #2649
Open
1 of 2 tasks

Enable filtering by year for EIA 861 and FERC 714 ETLs #2628

e-belfer opened this issue Jun 5, 2023 · 0 comments · Fixed by #2649
Labels
dagster Issues related to our use of the Dagster orchestrator eia861 Anything having to do with EIA Form 861 ferc714 Anything having to do with FERC Form 714
Milestone

Comments

@e-belfer
Copy link
Member

e-belfer commented Jun 5, 2023

Right now the EIA 861 and FERC 714 interim ETLs process data for all years for both the full and the fast ETL. As we add more of this data into dagster, and notably as we incorporate the service territory analyses (#2437), the inability to subset this data by year is causing memory issues for the CI.

To make this change, we would need to do the following:

@e-belfer e-belfer moved this from New to Backlog in Catalyst Megaproject Jun 5, 2023
@zaneselvans zaneselvans added ferc714 Anything having to do with FERC Form 714 eia861 Anything having to do with EIA Form 861 dagster Issues related to our use of the Dagster orchestrator labels Jun 5, 2023
@e-belfer e-belfer added this to the 2023 Spring milestone Jun 5, 2023
@e-belfer e-belfer linked a pull request Jun 9, 2023 that will close this issue
8 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dagster Issues related to our use of the Dagster orchestrator eia861 Anything having to do with EIA Form 861 ferc714 Anything having to do with FERC Form 714
Projects
Status: Backlog
Development

Successfully merging a pull request may close this issue.

2 participants