Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

What to do with new datasets? #123

Open
Dopervigest opened this issue Oct 25, 2022 · 1 comment
Open

What to do with new datasets? #123

Dopervigest opened this issue Oct 25, 2022 · 1 comment
Labels
doubts help wanted Extra attention is needed question Further information is requested

Comments

@Dopervigest
Copy link
Collaborator

New datasets must be checked for availability and integrity. In order to do this, we have download tests that check if each URL is reachable, and the files they provide correspond to the ones that we intend them to.
The checks work great but consume quite a bit of time. The files I added in PR "Added some peices of miccai2015 dataset" each take around a minute to download, so i didn't commit the tests for now.
I don't know how to resolve this problem, the files are already compressed with 7z and there are only 10 models in both datasets. If you have any suggestions on what i should do, i'd be very glad to hear them

@Dopervigest Dopervigest added help wanted Extra attention is needed question Further information is requested doubts labels Oct 25, 2022
@KonstantinUshenin
Copy link
Collaborator

@Dopervigest , @maksim-dzhigil You should make another folders for long test, like test_long. When we separate main and develop, some test can be run on push to develop and some on push to main.

Datasets availability will be checked only on PR to the main, but not to the develop. Also, we will pay for CDN, and files can be loaded faster.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
doubts help wanted Extra attention is needed question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants