Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Recording Oracle] Appending New Chunks #2490

Open
portuu3 opened this issue Sep 6, 2024 · 0 comments
Open

[Recording Oracle] Appending New Chunks #2490

portuu3 opened this issue Sep 6, 2024 · 0 comments
Milestone
Batch processing

Comments

@portuu3
Copy link
Collaborator

portuu3 commented Sep 6, 2024

Description
Modify the Recording Oracle to append new chunks of data into the intermediate results file in cloud storage.

Motivation
Enable processing jobs by chunks.

Implementation details

  • Update the Recording Oracle to write new chunks to an existing file.
  • Use logical identifiers to separate each chunk within the file for easy access and identification.
@portuu3 portuu3 moved this to Icebox in Core-tech - 2024 Sep 9, 2024
@portuu3 portuu3 added this to the Batch processing milestone Sep 10, 2024
@portuu3 portuu3 moved this from Icebox to Backlog in Core-tech - 2024 Oct 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
Core-tech - 2025
Status: No status
Milestone
Batch processing
Development

No branches or pull requests
1 participant
@portuu3