Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add parquet data source for integration testing #55

Open
Tracked by #260
jtcohen6 opened this issue Nov 29, 2020 · 0 comments
Open
Tracked by #260

Add parquet data source for integration testing #55

jtcohen6 opened this issue Nov 29, 2020 · 0 comments
Labels
enhancement New feature or request tech_debt

Comments

@jtcohen6
Copy link
Collaborator

Describe the feature

We've got people.csv and people.json, why not people.parquet?

Describe alternatives you've considered

Other formats we could consider adding, recognizing that they may not be supported on all databases:

  • avro
  • delta

Additional context

All databases should have support for parquet, since it's the canonical open source column-store format

Who will this benefit?

While this package can be used to stage CSV and JSON sources, the optimal production use case is columnar file formats + analytic databases/lakes/processing technologies. We should test it accordingly.

@jtcohen6 jtcohen6 added the enhancement New feature or request label Nov 29, 2020
@dataders dataders mentioned this issue Mar 8, 2024
@dataders dataders added this to the Upgrade CI 2.0 milestone Apr 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request tech_debt
Projects
None yet
Development

No branches or pull requests

2 participants