Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Choose an example output format that is Python friendly #74

Open
tgadam opened this issue Jul 12, 2022 · 4 comments
Open

Choose an example output format that is Python friendly #74

tgadam opened this issue Jul 12, 2022 · 4 comments
Assignees

Comments

@tgadam
Copy link
Contributor

tgadam commented Jul 12, 2022

Once a user gets fermentation into FSML then we want to be able to convert FSML to a Python (pandas) optimized format. One thought is Feather since it is optimized for Python, R, and Julia but it's not clear if that's the best first example.

https://pandas.pydata.org/docs/user_guide/io.html

Depends on #88

@juanneilson
Copy link
Member

TOML is being used a lot now in Python and is very versatile and human-readable.

@tgadam
Copy link
Contributor Author

tgadam commented Jul 12, 2022

I think one of the topics we need to focus on I think is performance and being data science friendly. https://towardsdatascience.com/big-data-file-formats-explained-dfaabe9e8b33

@tgadam
Copy link
Contributor Author

tgadam commented Jul 19, 2022

Reviewing applicable formats is still in progress.

@tgadam
Copy link
Contributor Author

tgadam commented Aug 2, 2022

@AndresPerezTesela indicated that we should first either generate or find a large dataset to try out with various output formats.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants