Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Populate development DB(s) with test fixture data #2313

Open
Jongmassey opened this issue Jan 20, 2025 · 1 comment
Open

Populate development DB(s) with test fixture data #2313

Jongmassey opened this issue Jan 20, 2025 · 1 comment
Labels
deck-scrubbing Tech debt or other between-initiative tidy-up work

Comments

@Jongmassey
Copy link
Contributor

Following from discussion in #2117 it seems like using the test fixture data as dummy data for development is an attractive alternative to using a copy of the production database.

The OpenCodelists tests contain a number of fixtures (e.g. here but NB there exist fixtures in other apps) which are small, internally consistent, comprehensive (in terms of coding systems and site features), and devoid of user PII.

The comprehensiveness of these test fixtures ensures adequate test coverage, but also makes them potentially useful for interactively and iteratively developing new features/fixing bugs.

Being PyTest fixtures, these are loaded into in-memory sqlite databases which are created at the beginning of a test session and destroyed at the end. To be useful as development data, a process to load them into persistent on-disk sqlite databases would be required.

Directly calling the test fixtures outside of a PyTest test is not permitted so some other way around this may be required (see this comment for relevant prior art in ehrQL).

This work may or may not be affected to changes to our test database configuration required for upgrading to Django 5.1.x in #2115

@Jongmassey Jongmassey added the deck-scrubbing Tech debt or other between-initiative tidy-up work label Jan 20, 2025
@Jongmassey
Copy link
Contributor Author

Alternatively, here's a POC of how to remove PII from the backups which would make them suitable for developer use.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
deck-scrubbing Tech debt or other between-initiative tidy-up work
Projects
None yet
Development

No branches or pull requests

1 participant