Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RHCLOUD-36102] - Slow loading of data from roles endpoint perfo… #1354

Conversation

EvanCasey13
Copy link
Contributor

@EvanCasey13 EvanCasey13 commented Nov 29, 2024

…rmance fixes

Link(s) to Jira

https://issues.redhat.com/browse/RHCLOUD-36102

Description of Intent of Change(s)

Originally focused on fine grained handling of filtering and pagination, this did not lead to any significant reduction in number of queries or performance improvements.

Using prefetch_related for "access", "ext_relation" and "access__permission" reduced the number of queries from 33 to 6 for a request to the roles endpoint for the get_role_queryset method.

Local Testing

How can the feature be exercised?
How can the bug be exploited and fix confirmed?
Is any special local setup required?

Checklist

  • if API spec changes are required, is the spec updated?
  • are there any pre/post merge actions required? if so, document here.
  • are theses changes covered by unit tests?
  • if warranted, are documentation changes accounted for?
  • does this require migration changes?
    • if yes, are they backwards compatible?
  • is there known, direct impact to dependent teams/components?
    • if yes, how will this be handled?

Secure Coding Practices Checklist Link

Secure Coding Practices Checklist

  • Input Validation
  • Output Encoding
  • Authentication and Password Management
  • Session Management
  • Access Control
  • Cryptographic Practices
  • Error Handling and Logging
  • Data Protection
  • Communication Security
  • System Configuration
  • Database Security
  • File Management
  • Memory Management
  • General Coding Practices

@EvanCasey13 EvanCasey13 changed the title [WIP][RHCLOUD-36102] - Slow loading of data from roles endpoint perfo… [RHCLOUD-36102] - Slow loading of data from roles endpoint perfo… Dec 9, 2024
Copy link
Contributor

@astrozzc astrozzc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on the extra query you got from django silk, this seems to be n+1 query issue.
I think you can improve the performance by using prefetch related in the get_role_queryset method. We just need to include the permission lookup and external relation lookup there as well.

rbac/management/role/serializer.py Outdated Show resolved Hide resolved
rbac/management/role/view.py Outdated Show resolved Hide resolved
tests/management/role/test_view.py Outdated Show resolved Hide resolved
tests/management/role/test_view.py Outdated Show resolved Hide resolved
rbac/management/role/view.py Outdated Show resolved Hide resolved
@EvanCasey13 EvanCasey13 marked this pull request as draft December 12, 2024 16:32
@EvanCasey13 EvanCasey13 marked this pull request as ready for review December 19, 2024 15:23
@EvanCasey13 EvanCasey13 marked this pull request as draft January 13, 2025 16:27
@EvanCasey13 EvanCasey13 marked this pull request as ready for review January 14, 2025 10:55
@EvanCasey13
Copy link
Contributor Author

/retest

1 similar comment
@petracihalova
Copy link
Contributor

/retest

Copy link
Contributor

@astrozzc astrozzc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Not sure if the default limit would change any behavior though

@@ -141,6 +142,7 @@ class RoleViewSet(
filterset_class = RoleFilter
ordering_fields = ("name", "display_name", "modified", "policyCount")
ordering = ("name",)
default_limit = StandardResultsSetPagination.default_limit
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this intended?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope thats redundant removed now, was from my testing of filtering and pagination stuff

Copy link
Contributor

@astrozzc astrozzc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Not sure if the default limit would change any behavior though. Need a second eye

@petracihalova petracihalova merged commit edf766e into RedHatInsights:master Jan 30, 2025
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants