Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Issue #3851] Delete historical data from our tables #3954

Closed

Conversation

mikehgrantsgov
Copy link
Collaborator

Summary

Fixes #3851

Time to review: 15 mins

Changes proposed

Add SQL transaction script to remove historical data from our environments.
Update seed script to generate historical data to test deletion (this can be removed in a follow up ticket).

Context for reviewers

Additional context in ticket, but remove versioned records from the following tables:
opportunity_summary
link_opportunity_summary_funding_instrument
link_opportunity_summary_funding_category
link_opportunity_summary_applicant_type

Tested working in a local environment.

Additional information

Leaving this in draft status for additional review.


-- If everything looks good, commit the transaction
-- If not, ROLLBACK instead
COMMIT;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would make the ROLLBACK uncommented and the COMMIT require the extra effort, just in case the script gets run end-to-end by accident.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense!

@mikehgrantsgov mikehgrantsgov changed the title Delete historical data from our tables [Issue #3851] Delete historical data from our tables Feb 24, 2025
@mikehgrantsgov mikehgrantsgov marked this pull request as ready for review February 24, 2025 16:57
Copy link
Collaborator

@chouinar chouinar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the SQL looks good - I don't think we need to check in these changes (also avoids needing to undo the seed script). The SQL / script won't even function once we remove the revision number column in the next ticket after we run all of this.

@mikehgrantsgov
Copy link
Collaborator Author

I think the SQL looks good - I don't think we need to check in these changes (also avoids needing to undo the seed script). The SQL / script won't even function once we remove the revision number column in the next ticket after we run all of this.

Makes sense. Let me know when you want to pair on running these and we can close the PR / leave it unmerged.

@chouinar
Copy link
Collaborator

@mikehgrantsgov - apologies on missing your message earlier - happy to pair on this.

One caveat to your script from some recent stuff I've run - our current way of running adhoc queries against the db via the RDS query editor doesn't actually let you run transactions. The editor itself either auto-commits every command, or at the end of every command. There's no option to control the start/end of the transaction yourself.

@mikehgrantsgov
Copy link
Collaborator Author

Closing PR as we will not merge and DELETEs have been done in all envs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Delete historical data from our tables in each environment
3 participants