Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Huggingface dataconfig #89

Merged
merged 9 commits into from
Oct 31, 2023
Merged

Huggingface dataconfig #89

merged 9 commits into from
Oct 31, 2023

Conversation

dfulu
Copy link
Member

@dfulu dfulu commented Oct 24, 2023

Pull Request

Add functionality so that the data config will be uploaded to huggingface alongside the model config and weights. This should make it easier to keep the datapipe and model aligned.

  • Added to training script so when model is trained the data config is saved with the model checkpoints
  • Added to base_model and checkpoint_to_huggingface so when saving the model, the data config is saved alongside the model weights and the model config. The data config is stripped of local paths before uploading.
  • Add function which can be used to pull the model config from huggingface. This will be used from the app

Checklist:

  • My code follows OCF's coding style guidelines
  • I have performed a self-review of my own code
  • I have made corresponding changes to the documentation
  • I have added tests that prove my fix is effective or that my feature works
  • I have checked my code and corrected any misspellings

Copy link

codecov bot commented Oct 31, 2023

Codecov Report

Merging #89 (a243308) into main (0ba2c1d) will decrease coverage by 0.68%.
Report is 13 commits behind head on main.
The diff coverage is 18.75%.

@@            Coverage Diff             @@
##             main      #89      +/-   ##
==========================================
- Coverage   50.55%   49.87%   -0.68%     
==========================================
  Files          23       23              
  Lines        1614     1644      +30     
==========================================
+ Hits          816      820       +4     
- Misses        798      824      +26     
Files Coverage Δ
pvnet/__init__.py 100.00% <100.00%> (ø)
pvnet/models/multimodal/multimodal.py 95.60% <ø> (ø)
pvnet/models/multimodal/site_encoders/encoders.py 95.83% <ø> (ø)
pvnet/training.py 0.00% <0.00%> (ø)
pvnet/models/base_model.py 33.33% <20.00%> (-2.16%) ⬇️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@dfulu dfulu marked this pull request as ready for review October 31, 2023 16:26
@dfulu dfulu requested a review from jacobbieker October 31, 2023 16:30
@dfulu dfulu merged commit b922c9d into main Oct 31, 2023
2 of 4 checks passed
@dfulu dfulu deleted the huggingface_dataconfig branch October 31, 2023 16:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants