Make a multi-year EIA MECS archive #542
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
cmgosnell
changed the title
cmgosnell
commented
Jan 24, 2025
Comment on lines
37
to
56
| if int(year) >= 2006: | ||
| table_link_pattern = re.compile( | ||
| r"(RSE|)[Tt]able(\d{1,2}|\d{1.1})_(\d{1,2})(.xlsx|.xls)" | ||
| ) | ||
| elif int(year) == 2002: | ||
| table_link_pattern = re.compile( | ||
| r"(RSE|)[Tt]able(\d{1,2}).(\d{1,2})_\d{1,2}(.xlsx|.xls)" | ||
| ) | ||
| elif int(year) == 1998: | ||
| table_link_pattern = re.compile( | ||
| r"(d|e)\d{2}([a-z]\d{1,2})_(\d{1,2})(.xlsx|.xls)" | ||
| ) | ||
| elif int(year) == 1994: | ||
| # These earlier years the pattern is functional but not actually that informative. | ||
| # so we will just use the original name by making the whole pattern a match | ||
| table_link_pattern = re.compile( | ||
| r"((rse|)m\d{2}_(\d{2})([a-d]|)(.xlsx|.xls))" | ||
| ) | ||
| elif int(year) == 1991: | ||
| table_link_pattern = re.compile(r"((rse|)mecs(\d{2})([a-z]|)(.xlsx|.xls))") |
There was a problem hiding this comment.
I don't love this whole situation but i didn't know what else to do. i thought about making a dict w/ year as key and pattern as value but then it wouldn't naturally grab the next year if the newer pattern holds.
There was a problem hiding this comment.
You could make the latest year be the default pattern and update with a dict-key if it's one of the other years? I agree it's a bit verbose.
e-belfer
requested changes
Jan 27, 2025
e-belfer
reviewed
Jan 27, 2025
e-belfer
approved these changes
Jan 28, 2025
|
published archives: |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Overview
Closes #516.
converted the 2018 mecs archive into a mutli-year archive. the link patterns for each of the years before 2006 are pretty different. The info about the major and minor table numbers are not in the original file names for the 1998 and 1994 archives so I didn't attempted to rename those.
Testing
How did you make sure this worked? How can a reviewer verify this?
Unpublished archives:
https://zenodo.org/uploads/14749820
https://sandbox.zenodo.org/uploads/158873
To-do list
Tasks