Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make it possible to pass another sources dict to DataSource #4003

Merged
merged 2 commits into from
Jan 7, 2025
Merged

Make it possible to pass another sources dict to DataSource #4003

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
e04ffb7
Make it possible to pass another sources dict to DataSource
e-belfer Jan 7, 2025
6f3100e
Remove straggler PHMSA reference from other branch
e-belfer Jan 7, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 8 additions & 4 deletions src/pudl/metadata/classes.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1035,20 +1035,22 @@ def to_rst(
sys.stdout.write(rendered)

@classmethod
def from_field_namespace(cls, x: str) -> list["DataSource"]:
def from_field_namespace(
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't get called in pudl-archiver, so I could revert this change. Just figured for consistency it'd make sense to have the same behavior as below.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think consistency here is good

cls, x: str, sources: dict[str, Any] = SOURCES
) -> list["DataSource"]:
"""Return list of DataSource objects by field namespace."""
return [
cls(**cls.dict_from_id(name))
for name, val in SOURCES.items()
for name, val in sources.items()
if val.get("field_namespace") == x
]

@staticmethod
def dict_from_id(x: str) -> dict:
def dict_from_id(x: str, sources: dict[str, Any] = SOURCES) -> dict:
"""Look up the source by source name in the metadata."""
# If ID ends with _xbrl strip end to find data source
lookup_id = x.replace("_xbrl", "")
return {"name": x, **copy.deepcopy(SOURCES[lookup_id])}
return {"name": x, **copy.deepcopy(sources[lookup_id])}

@classmethod
def from_id(cls, x: str) -> "DataSource":
Expand Down Expand Up @@ -1285,6 +1287,7 @@ class Resource(PudlMeta):
"pudl",
"nrelatb",
"vcerare",
"phmsagas",
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops! This crossed over from another branch, I'll remove this.

]
| None
) = None
Expand Down Expand Up @@ -1313,6 +1316,7 @@ class Resource(PudlMeta):
"service_territories",
"nrelatb",
"vcerare",
"phmsagas",
]
| None
) = None
Expand Down
Loading