Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Check for empty data files #31

Open
franTarkenton opened this issue Oct 22, 2024 · 0 comments
Open

Check for empty data files #31

franTarkenton opened this issue Oct 22, 2024 · 0 comments
Assignees
Labels
enhancement New feature or request

Comments

@franTarkenton
Copy link
Member

Currently when running the ingest process it will check to see if a file already exists locally. If it does then the script will skip over that file and now pull it from object storage.

The check should also make sure that the file is valid, not 0 bytes in size, and has at least on row of data. if not then it should be re-downloaded.

@franTarkenton franTarkenton added the enhancement New feature or request label Oct 22, 2024
@franTarkenton franTarkenton self-assigned this Oct 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Status: New
Development

No branches or pull requests

1 participant