Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Codspeed performance benchmarks #1831

Merged
merged 9 commits into from
Dec 30, 2024
Merged

Conversation

henadzit
Copy link
Contributor

@henadzit henadzit commented Dec 29, 2024

Description

Motivation and Context

Codspeed should help us tracking performance regression and improvement. It lets you see the time difference introduced by commits but also see how long each method or function takes. I'm planning to do some performance related work soon, so it should be helpful. Codspeed is used by other large open-source projects, for instance, pydantic.

Here's a report example https://codspeed.io/henadzit/tortoise-orm/branches/feat%2Fcodspeed, although it lacks the comparison with the baseline because, well, there is no baseline yet. It should start appearing after this PR is merged. This example is from PR in my fork.

Here's an example of Model.get profiling (we can already see that 20% of time is spent in pypika which is crazy):
Screenshot 2024-12-29 at 08 54 19

How Has This Been Tested?

make ci
pytest tests/benchmarks --codspeed

Checklist:

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have added the changelog accordingly.
  • I have read the CONTRIBUTING document.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

@coveralls
Copy link

coveralls commented Dec 29, 2024

Pull Request Test Coverage Report for Build 12535084212

Details

  • 6 of 6 (100.0%) changed or added relevant lines in 1 file are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.002%) to 90.253%

Totals Coverage Status
Change from base Build 12534043192: 0.002%
Covered Lines: 6405
Relevant Lines: 6983

💛 - Coveralls

@henadzit
Copy link
Contributor Author

@abondar could you please add the repo to https://codspeed.io/? Here's the guide. I don't seem to have enough permissions. Lack of the CODSPEED_TOKEN secret is the reason why the Codspeed check is failing right now.

@abondar
Copy link
Member

abondar commented Dec 29, 2024

@henadzit I am not sure how to proceed with it - I added repository to list of repos in codespeed, but the moment it should have shown me popup with token to copy - popup immediately appeared and disappeared and I didn't have a chance to note down the token.

But it doesn't allow me to resetup integration again, as it says "Disabling repository is not supported yet"

@henadzit
Copy link
Contributor Author

@henadzit I am not sure how to proceed with it - I added repository to list of repos in codespeed, but the moment it should have shown me popup with token to copy - popup immediately appeared and disappeared and I didn't have a chance to note down the token.

But it doesn't allow me to resetup integration again, as it says "Disabling repository is not supported yet"

You should be able to find the token in Settings. Here's how it looks in my fork:

Screenshot 2024-12-29 at 20 18 44

Copy link

codspeed-hq bot commented Dec 30, 2024

CodSpeed Performance Report

Congrats! CodSpeed is installed 🎉

🆕 5 new benchmarks were detected.

You will start to see performance impacts in the reports once the benchmarks are run from your default branch.

Detected benchmarks

  • test_create_few_fields (524.8 µs)
  • test_create_many_fields (830.4 µs)
  • test_filter_few_fields (1.1 ms)
  • test_get_few_fields (749.5 µs)
  • test_get_many_fields (1.4 ms)

@abondar
Copy link
Member

abondar commented Dec 30, 2024

Seems like it works now

@henadzit henadzit merged commit be5bbc1 into tortoise:develop Dec 30, 2024
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants