Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enhance(pagination)!: remove total results calculation for performance improvement #1254

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

ecrupper
Copy link
Contributor

@ecrupper ecrupper commented Feb 7, 2025

Rather than calculate pagination logic on total count, we can leverage whether or not the current page of results is equal to the per-page setting.

This will create some awkward situations where the total results are a multiple of the per-page setting, but we can handle this in the UI more gracefully (there will need to be a follow up PR in that repo).

However, the awkward handling is worth the performance improvement. Using telemetry, we have been able to discover that the most expensive DB queries in the application are all the counts. They are not necessary for functionality and are notoriously not performant. This change should make almost every list API far more snappy.

@ecrupper ecrupper requested a review from a team as a code owner February 7, 2025 17:00
Copy link

codecov bot commented Feb 7, 2025

Codecov Report

Attention: Patch coverage is 52.13675% with 56 lines in your changes missing coverage. Please review.

Project coverage is 56.81%. Comparing base (cf8aea3) to head (3a6a89b).

Files with missing lines Patch % Lines
api/pagination.go 0.00% 3 Missing ⚠️
secret/native/list.go 25.00% 3 Missing ⚠️
api/build/graph.go 0.00% 2 Missing ⚠️
api/build/list_org.go 0.00% 2 Missing ⚠️
api/build/list_repo.go 0.00% 2 Missing ⚠️
api/build/update.go 0.00% 2 Missing ⚠️
api/hook/list.go 0.00% 2 Missing ⚠️
api/log/list_build.go 0.00% 2 Missing ⚠️
api/pipeline/list.go 0.00% 2 Missing ⚠️
api/repo/list.go 0.00% 2 Missing ⚠️
... and 24 more
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1254      +/-   ##
==========================================
+ Coverage   56.59%   56.81%   +0.22%     
==========================================
  Files         629      629              
  Lines       35645    35411     -234     
==========================================
- Hits        20172    20120      -52     
+ Misses      14792    14662     -130     
+ Partials      681      629      -52     
Files with missing lines Coverage Δ
database/build/list.go 89.28% <100.00%> (+14.28%) ⬆️
database/hook/list.go 89.65% <100.00%> (+13.97%) ⬆️
database/log/list.go 90.00% <100.00%> (+13.68%) ⬆️
database/pipeline/list.go 81.25% <100.00%> (+11.25%) ⬆️
database/repo/list.go 90.90% <100.00%> (+12.86%) ⬆️
database/schedule/list.go 89.65% <100.00%> (+13.97%) ⬆️
database/schedule/list_active.go 90.00% <100.00%> (+13.68%) ⬆️
database/secret/list.go 90.32% <100.00%> (+13.39%) ⬆️
database/service/list.go 86.95% <100.00%> (+15.98%) ⬆️
database/step/list.go 86.95% <100.00%> (+15.98%) ⬆️
... and 35 more

@ecrupper ecrupper changed the title enhance(pagination): remove total results calculation for performance improvement enhance(pagination)!: remove total results calculation for performance improvement Feb 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant