Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add support for splitting input and output data sources in DPK #100

Open
blublinsky opened this issue Feb 6, 2025 · 1 comment
Open

Add support for splitting input and output data sources in DPK #100

blublinsky opened this issue Feb 6, 2025 · 1 comment
Assignees
@blublinsky

Comments

@blublinsky
Copy link
Contributor

Currently, DPK allows the definition of only one data source for both the input and output of the transformer
This new feature will allow for separate definitions for input and output for processing so that for example, you can transform data from S3 and write it HF and vice versa
@blublinsky blublinsky self-assigned this Feb 6, 2025
@blublinsky
Copy link
Contributor Author

PR is here and is close to completion. Final testing

@deanwampler deanwampler moved this to In Progress in FA5: OTDI Tasks Feb 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@blublinsky blublinsky

Labels
None yet
Projects
FA5: OTDI Tasks
Status: In Progress
Milestone
No milestone
Development

No branches or pull requests
1 participant
@blublinsky