Hf data access

[blublinsky]

Why are these changes needed?

AI Alliance is planning to use Data Prep Kit for validating data stored in HF datasets

Related issue number (if any).

#964

[touma-I]

Plese create an issue describing in details the problem you are trying to solve and provide outline of the sollution, solicit peer input before submitting a PR.

[deanwampler]

Plese create an issue describing in details the problem you are trying to solve and provide outline of the sollution, solicit peer input before submitting a PR.

Creating an issue is fine; lots of projects like requiring one for PRs. However, telling the user to "solicit peer input before submitting a PR" will just discourage people from submitting PRs. If you want to suggest that users "float a crazy idea" first, okay, but most people will just walk away if you expect this engagement. They are doing you a service by offering improvements.

Concerning this PR, integration with the HF dataset API is mandatory for DPK to be useful for OTDI, so please take this seriously.

[touma-I]

Plese create an issue describing in details the problem you are trying to solve and provide outline of the sollution, solicit peer input before submitting a PR.

Creating an issue is fine; lots of projects like requiring one for PRs. However, telling the user to "solicit peer input before submitting a PR" will just discourage people from submitting PRs. If you want to suggest that users "float a crazy idea" first, okay, but most people will just walk away if you expect this engagement. They are doing you a service by offering improvements.

Concerning this PR, integration with the HF dataset API is mandatory for DPK to be useful for OTDI, so please take this seriously.

@deanwampler thanks for the feedback. Peer review will allow us to determine if the propose enhancement will indeed improve the overall framework and therefore adoption. I don't know if this falls into the "crazy idea" you are talking about and will take a look at the issue once you have created one. Please be detailed in describing what you are trying to achieve and discuss in details the alternatives to reach your goal.