-
Notifications
You must be signed in to change notification settings - Fork 111
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
DATA-3441 Update data export command #4596
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looking pretty good, had a couple comments.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🚢
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🔥
This PR replaces our old
data export
command with two separate subcommands:data export binary
anddata export tabular
. The logic poweringdata export binary
is the same.data export tabular
was essentially rewritten and now usesExportTabularData
.This method:
--destination
.ExportTabularData
to query for a specific data source within an optional interval.data.ndjson
.If there is an error, the export process will be attempted up to 5 times. If there is still an error, the
data.ndjson
file is removed.Automated Testing:
TestTabularDataByFilterAction
to work with the new setup.data.ndjson
.Manual Testing:
Ran export command locally after setting my base-url to my
ExportTabularData
deployed branch.Queried for same data source as in DATA-3440
data.ndjson
file (30.7 MB).Also tested with interval start/end, just interval start, and just interval end. Confirmed that correct data ranges were returned ✅