Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Performance Test Set-up (1) #197

Merged
merged 36 commits into from
Nov 12, 2024
Merged

Conversation

louiseschmidtgen
Copy link
Contributor

@louiseschmidtgen louiseschmidtgen commented Nov 5, 2024

Performance Test

The performance test spins up three lxd containers that join a cluster. Currently, this cluster consists of three control planes. In a follow-up to this PR, kube-burner will run a load-test on the cluster and the resulting metrics are to be displayed as a comment on any future opened PRs.

Set-up

This PR sets up the harness and gh action for the performance test.

These k8s-dqlite combinations get tested on every test:

k8s-dqlite
head
base code
v1.1.11
v1.2.0

For simplification, we could opt to run the test on v1.1.11 and v1.2.0 only with the published snap (not swapping out the dqlite version).

Note

Tests are not run as a matrix but in sequence as the collected metrics will be turned into graphs at the end of the workflow.

This test set-up is similar to the k8s-snap integration test. The test is not part of that suite as it is designed as a performance measurement tool of the k8s-dqlite code. It is a tool to directly compare the performance improvements in PRs against the code in master. The test could be extended for microk8s. For those reasons the correct location for this test is in k8s-dqlite.

Copy link

github-actions bot commented Nov 5, 2024

Benchmark Result

goos: linux
goarch: amd64
pkg: github.com/canonical/k8s-dqlite/test
cpu: AMD EPYC 7763 64-Core Processor                
                                        │     Before     │                After                 │
                                        │     sec/op     │     sec/op      vs base              │
Compaction/sqlite-4                        19.81µ ±   8%   19.70µ ±    8%       ~ (p=0.710 n=7)
Compaction/dqlite-4                        41.47µ ±  10%   41.07µ ±    5%       ~ (p=0.902 n=7)
Create/sqlite/1-workers-4                  503.3µ ±   3%   484.7µ ±    8%  -3.69% (p=0.001 n=7)
Create/sqlite/4-workers-4                  495.3µ ±  10%   459.6µ ±   10%  -7.22% (p=0.026 n=7)
Create/sqlite/16-workers-4                 442.6µ ±   7%   430.8µ ±   19%       ~ (p=0.710 n=7)
Create/sqlite/64-workers-4                 482.8µ ±  14%   461.5µ ± 1060%       ~ (p=0.710 n=7)
Create/sqlite/128-workers-4                576.4µ ± 267%   737.5µ ±   48%       ~ (p=0.456 n=7)
Create/dqlite/1-workers-4                  1.148m ±   3%   1.055m ±    3%  -8.14% (p=0.001 n=7)
Create/dqlite/4-workers-4                 1017.9µ ±   6%   968.5µ ±    2%       ~ (p=0.053 n=7)
Create/dqlite/16-workers-4                 978.7µ ±   6%   952.8µ ±    1%       ~ (p=0.620 n=7)
Create/dqlite/64-workers-4                 974.0µ ±   4%   965.3µ ±    1%       ~ (p=0.128 n=7)
Create/dqlite/128-workers-4                983.3µ ±   4%   962.2µ ±    6%       ~ (p=0.259 n=7)
Delete/sqlite/1-workers-4                  508.5µ ±  18%   461.2µ ±    4%  -9.31% (p=0.001 n=7)
Delete/sqlite/4-workers-4                  471.7µ ±   3%   453.6µ ±    4%  -3.83% (p=0.017 n=7)
Delete/sqlite/16-workers-4                 451.5µ ±   9%   439.0µ ±    5%       ~ (p=0.097 n=7)
Delete/sqlite/64-workers-4                 490.8µ ±  19%   447.5µ ±   72%       ~ (p=0.383 n=7)
Delete/sqlite/128-workers-4                580.3µ ±  89%   612.7µ ±   73%       ~ (p=1.000 n=7)
Delete/dqlite/1-workers-4                  1.088m ±   2%   1.061m ±    6%       ~ (p=0.383 n=7)
Delete/dqlite/4-workers-4                  996.4µ ±   4%   961.3µ ±    3%  -3.52% (p=0.038 n=7)
Delete/dqlite/16-workers-4                 966.2µ ±   2%   967.2µ ±    3%       ~ (p=0.456 n=7)
Delete/dqlite/64-workers-4                 958.7µ ±   4%   949.3µ ±    3%       ~ (p=0.318 n=7)
Delete/dqlite/128-workers-4                956.7µ ±   6%   918.4µ ±   12%       ~ (p=0.620 n=7)
Get/sqlite-4                               239.9µ ±   1%   232.3µ ±    1%  -3.15% (p=0.001 n=7)
Get/dqlite-4                               359.2µ ±   1%   354.1µ ±    1%  -1.44% (p=0.001 n=7)
List/sqlite-tiny/all-4                     3.971µ ±   7%   3.932µ ±    7%       ~ (p=0.535 n=7)
List/sqlite-tiny/pagination-4              8.245µ ±   2%   8.149µ ±    2%       ~ (p=0.073 n=7)
List/sqlite-fits-in-page/all-4             7.838µ ±   8%   7.585µ ±    4%       ~ (p=0.383 n=7)
List/sqlite-fits-in-page/pagination-4      22.15µ ±   4%   21.29µ ±    4%  -3.90% (p=0.011 n=7)
List/sqlite-overflows-page/all-4           17.00µ ±   4%   17.00µ ±    3%       ~ (p=0.476 n=7)
List/sqlite-overflows-page/pagination-4    41.57µ ±   7%   42.92µ ±    6%       ~ (p=0.209 n=7)
List/dqlite-tiny/all-4                     2.936µ ±   7%   2.975µ ±    2%       ~ (p=0.056 n=7)
List/dqlite-tiny/pagination-4              6.088µ ±   1%   6.065µ ±    1%  -0.38% (p=0.017 n=7)
List/dqlite-fits-in-page/all-4             6.626µ ±   4%   6.634µ ±    6%       ~ (p=0.902 n=7)
List/dqlite-fits-in-page/pagination-4      17.47µ ±   3%   17.40µ ±    2%       ~ (p=0.620 n=7)
List/dqlite-overflows-page/all-4           21.11µ ±   2%   21.32µ ±    2%  +1.00% (p=0.007 n=7)
List/dqlite-overflows-page/pagination-4    44.56µ ±   3%   45.09µ ±    5%       ~ (p=0.620 n=7)
Update/sqlite/1-workers-4                  512.3µ ±  10%   510.5µ ±    3%       ~ (p=1.000 n=7)
Update/sqlite/4-workers-4                  499.1µ ±   2%   479.9µ ±   19%       ~ (p=0.259 n=7)
Update/sqlite/16-workers-4                 473.7µ ±   6%   453.8µ ±    8%       ~ (p=0.535 n=7)
Update/sqlite/64-workers-4                 500.5µ ± 991%   497.4µ ±   40%       ~ (p=0.805 n=7)
Update/sqlite/128-workers-4                796.6µ ±  59%   725.5µ ±  175%       ~ (p=1.000 n=7)
Update/dqlite/1-workers-4                  1.083m ±   9%   1.077m ±    5%       ~ (p=0.805 n=7)
Update/dqlite/4-workers-4                  964.4µ ±   8%   950.7µ ±    3%       ~ (p=0.535 n=7)
Update/dqlite/16-workers-4                 976.0µ ±   4%   950.8µ ±    6%       ~ (p=0.097 n=7)
Update/dqlite/64-workers-4                 989.4µ ±   5%   978.5µ ±    9%       ~ (p=0.318 n=7)
Update/dqlite/128-workers-4               1034.3µ ±   7%   965.1µ ±   12%       ~ (p=0.318 n=7)
geomean                                    206.1µ          202.3µ          -1.83%

                                        │        Before        │                    After                     │
                                        │ page-cache-misses/op │ page-cache-misses/op  vs base                │
Compaction/sqlite-4                               0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Compaction/dqlite-4                               0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Create/sqlite/1-workers-4                         0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Create/sqlite/4-workers-4                         0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Create/sqlite/16-workers-4                        0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Create/sqlite/64-workers-4                        0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Create/sqlite/128-workers-4                       0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Create/dqlite/1-workers-4                         0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Create/dqlite/4-workers-4                         0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Create/dqlite/16-workers-4                        0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Create/dqlite/64-workers-4                        0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Create/dqlite/128-workers-4                       0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Delete/sqlite/1-workers-4                         0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Delete/sqlite/4-workers-4                         0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Delete/sqlite/16-workers-4                        0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Delete/sqlite/64-workers-4                        0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Delete/sqlite/128-workers-4                       0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Delete/dqlite/1-workers-4                         0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Delete/dqlite/4-workers-4                         0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Delete/dqlite/16-workers-4                        0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Delete/dqlite/64-workers-4                        0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Delete/dqlite/128-workers-4                       0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Get/sqlite-4                                      0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Get/dqlite-4                                      0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-tiny/all-4                            0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-tiny/pagination-4                     0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-fits-in-page/all-4                    0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-fits-in-page/pagination-4             0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-overflows-page/all-4                  0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-overflows-page/pagination-4           0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-tiny/all-4                            0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-tiny/pagination-4                     0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-fits-in-page/all-4                    0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-fits-in-page/pagination-4             0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-overflows-page/all-4                  0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-overflows-page/pagination-4           0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Update/sqlite/1-workers-4                         0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Update/sqlite/4-workers-4                         0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Update/sqlite/16-workers-4                        0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Update/sqlite/64-workers-4                        0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Update/sqlite/128-workers-4                       0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Update/dqlite/1-workers-4                         0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Update/dqlite/4-workers-4                         0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Update/dqlite/16-workers-4                        0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Update/dqlite/64-workers-4                        0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Update/dqlite/128-workers-4                       0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
geomean                                                      ²                         +0.00%               ²
¹ all samples are equal
² summaries must be >0 to compute geomean

                                        │        Before        │                    After                     │
                                        │ page-cache-spills/op │ page-cache-spills/op  vs base                │
Compaction/sqlite-4                               0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Compaction/dqlite-4                               0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Create/sqlite/1-workers-4                         0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Create/sqlite/4-workers-4                         0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Create/sqlite/16-workers-4                        0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Create/sqlite/64-workers-4                        0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Create/sqlite/128-workers-4                       0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Create/dqlite/1-workers-4                         0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Create/dqlite/4-workers-4                         0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Create/dqlite/16-workers-4                        0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Create/dqlite/64-workers-4                        0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Create/dqlite/128-workers-4                       0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Delete/sqlite/1-workers-4                         0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Delete/sqlite/4-workers-4                         0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Delete/sqlite/16-workers-4                        0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Delete/sqlite/64-workers-4                        0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Delete/sqlite/128-workers-4                       0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Delete/dqlite/1-workers-4                         0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Delete/dqlite/4-workers-4                         0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Delete/dqlite/16-workers-4                        0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Delete/dqlite/64-workers-4                        0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Delete/dqlite/128-workers-4                       0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Get/sqlite-4                                      0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Get/dqlite-4                                      0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-tiny/all-4                            0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-tiny/pagination-4                     0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-fits-in-page/all-4                    0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-fits-in-page/pagination-4             0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-overflows-page/all-4                  0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-overflows-page/pagination-4           0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-tiny/all-4                            0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-tiny/pagination-4                     0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-fits-in-page/all-4                    0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-fits-in-page/pagination-4             0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-overflows-page/all-4                  0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-overflows-page/pagination-4           0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Update/sqlite/1-workers-4                         0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Update/sqlite/4-workers-4                         0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Update/sqlite/16-workers-4                        0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Update/sqlite/64-workers-4                        0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Update/sqlite/128-workers-4                       0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Update/dqlite/1-workers-4                         0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Update/dqlite/4-workers-4                         0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Update/dqlite/16-workers-4                        0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Update/dqlite/64-workers-4                        0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Update/dqlite/128-workers-4                       0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
geomean                                                      ²                         +0.00%               ²
¹ all samples are equal
² summaries must be >0 to compute geomean

                                        │    Before     │                After                │
                                        │ page-reads/op │ page-reads/op  vs base              │
Compaction/sqlite-4                         12.29 ±  1%     12.22 ±  1%  -0.57% (p=0.010 n=7)
Compaction/dqlite-4                         12.19 ±  1%     12.17 ±  1%       ~ (p=0.933 n=7)
Create/sqlite/1-workers-4                   13.87 ±  0%     13.88 ±  0%       ~ (p=0.319 n=7)
Create/sqlite/4-workers-4                   12.80 ±  3%     12.79 ±  3%       ~ (p=0.686 n=7)
Create/sqlite/16-workers-4                  10.63 ±  5%     10.78 ±  9%       ~ (p=0.438 n=7)
Create/sqlite/64-workers-4                  8.936 ±  7%     8.767 ± 32%       ~ (p=0.165 n=7)
Create/sqlite/128-workers-4                 8.326 ± 19%     8.294 ±  6%       ~ (p=0.535 n=7)
Create/dqlite/1-workers-4                   13.28 ±  2%     13.33 ±  1%       ~ (p=0.553 n=7)
Create/dqlite/4-workers-4                   10.97 ±  2%     10.89 ±  1%       ~ (p=0.197 n=7)
Create/dqlite/16-workers-4                  10.96 ±  1%     10.91 ±  1%       ~ (p=0.157 n=7)
Create/dqlite/64-workers-4                  10.94 ±  1%     10.91 ±  1%  -0.27% (p=0.048 n=7)
Create/dqlite/128-workers-4                 10.92 ±  1%     10.95 ±  1%       ~ (p=0.876 n=7)
Delete/sqlite/1-workers-4                   16.23 ±  0%     16.23 ±  0%       ~ (p=0.583 n=7)
Delete/sqlite/4-workers-4                   14.61 ±  3%     14.75 ±  4%       ~ (p=0.435 n=7)
Delete/sqlite/16-workers-4                  12.10 ±  5%     12.34 ±  5%       ~ (p=0.259 n=7)
Delete/sqlite/64-workers-4                  9.954 ±  6%    10.010 ±  6%       ~ (p=0.805 n=7)
Delete/sqlite/128-workers-4                 9.454 ±  4%     9.327 ±  2%  -1.34% (p=0.038 n=7)
Delete/dqlite/1-workers-4                   15.79 ±  1%     15.56 ±  1%  -1.46% (p=0.002 n=7)
Delete/dqlite/4-workers-4                   13.43 ±  1%     13.38 ±  1%       ~ (p=0.103 n=7)
Delete/dqlite/16-workers-4                  13.38 ±  1%     13.32 ±  1%       ~ (p=0.476 n=7)
Delete/dqlite/64-workers-4                  13.31 ±  2%     13.18 ±  1%       ~ (p=0.177 n=7)
Delete/dqlite/128-workers-4                 13.21 ±  2%     13.10 ±  2%  -0.83% (p=0.011 n=7)
Get/sqlite-4                                14.96 ±  0%     14.95 ±  0%       ~ (p=0.286 n=7)
Get/dqlite-4                                11.95 ±  0%     11.95 ±  0%       ~ (p=0.788 n=7)
List/sqlite-tiny/all-4                     191.5m ±  1%    191.3m ±  1%       ~ (p=0.645 n=7)
List/sqlite-tiny/pagination-4               1.131 ±  3%     1.131 ±  0%       ~ (p=1.000 n=7)
List/sqlite-fits-in-page/all-4             163.9m ±  2%    162.4m ±  2%       ~ (p=0.874 n=7)
List/sqlite-fits-in-page/pagination-4      963.6m ±  3%    962.3m ±  1%       ~ (p=0.515 n=7)
List/sqlite-overflows-page/all-4           177.1m ±  0%    177.1m ±  0%       ~ (p=1.000 n=7)
List/sqlite-overflows-page/pagination-4     1.404 ± 24%     1.399 ± 30%       ~ (p=0.469 n=7)
List/dqlite-tiny/all-4                     191.4m ±  0%    191.5m ±  0%       ~ (p=0.112 n=7)
List/dqlite-tiny/pagination-4               1.126 ±  0%     1.126 ±  3%       ~ (p=0.538 n=7)
List/dqlite-fits-in-page/all-4             164.1m ±  0%    163.8m ±  1%       ~ (p=0.221 n=7)
List/dqlite-fits-in-page/pagination-4      955.0m ±  3%    955.2m ±  3%       ~ (p=0.394 n=7)
List/dqlite-overflows-page/all-4           183.9m ±  0%    184.0m ±  0%       ~ (p=0.219 n=7)
List/dqlite-overflows-page/pagination-4     1.071 ±  1%     1.071 ±  3%       ~ (p=1.000 n=7)
Update/sqlite/1-workers-4                   15.83 ±  0%     15.81 ±  0%       ~ (p=0.092 n=7)
Update/sqlite/4-workers-4                   14.59 ±  3%     14.77 ±  3%       ~ (p=0.477 n=7)
Update/sqlite/16-workers-4                  12.83 ±  7%     12.62 ±  4%       ~ (p=0.456 n=7)
Update/sqlite/64-workers-4                  11.08 ±  8%     11.11 ±  7%       ~ (p=0.709 n=7)
Update/sqlite/128-workers-4                 10.57 ±  5%     10.62 ±  8%       ~ (p=0.403 n=7)
Update/dqlite/1-workers-4                   15.40 ±  1%     15.42 ±  1%       ~ (p=0.591 n=7)
Update/dqlite/4-workers-4                   12.72 ±  2%     12.78 ±  1%       ~ (p=0.783 n=7)
Update/dqlite/16-workers-4                  12.78 ±  1%     12.76 ±  1%       ~ (p=0.269 n=7)
Update/dqlite/64-workers-4                  12.87 ±  2%     12.95 ±  3%       ~ (p=0.402 n=7)
Update/dqlite/128-workers-4                 13.78 ±  4%     13.47 ±  4%       ~ (p=0.068 n=7)
geomean                                     5.197           5.189        -0.15%

                                        │     Before     │                 After                  │
                                        │ page-writes/op │ page-writes/op  vs base                │
Compaction/sqlite-4                       190.7m ±  0%        190.5m ± 0%       ~ (p=0.395 n=7)
Compaction/dqlite-4                       187.6m ±  1%        187.4m ± 0%       ~ (p=0.330 n=7)
Create/sqlite/1-workers-4                  4.356 ±  0%         4.360 ± 0%       ~ (p=0.129 n=7)
Create/sqlite/4-workers-4                  4.316 ±  0%         4.326 ± 1%       ~ (p=0.513 n=7)
Create/sqlite/16-workers-4                 4.293 ±  1%         4.298 ± 1%       ~ (p=0.135 n=7)
Create/sqlite/64-workers-4                 4.259 ±  1%         4.246 ± 5%       ~ (p=0.513 n=7)
Create/sqlite/128-workers-4                4.217 ±  4%         4.210 ± 2%       ~ (p=0.710 n=7)
Create/dqlite/1-workers-4                  4.216 ±  2%         4.266 ± 1%  +1.19% (p=0.001 n=7)
Create/dqlite/4-workers-4                  4.281 ±  1%         4.286 ± 0%       ~ (p=0.135 n=7)
Create/dqlite/16-workers-4                 4.283 ±  1%         4.297 ± 0%       ~ (p=0.366 n=7)
Create/dqlite/64-workers-4                 4.286 ±  0%         4.288 ± 0%       ~ (p=0.827 n=7)
Create/dqlite/128-workers-4                4.296 ±  2%         4.300 ± 0%       ~ (p=0.367 n=7)
Delete/sqlite/1-workers-4                  4.384 ±  0%         4.390 ± 0%       ~ (p=0.554 n=7)
Delete/sqlite/4-workers-4                  4.388 ±  1%         4.378 ± 1%       ~ (p=0.685 n=7)
Delete/sqlite/16-workers-4                 4.381 ±  1%         4.359 ± 1%       ~ (p=0.365 n=7)
Delete/sqlite/64-workers-4                 4.347 ±  1%         4.360 ± 1%       ~ (p=0.478 n=7)
Delete/sqlite/128-workers-4                4.365 ±  1%         4.348 ± 1%       ~ (p=0.221 n=7)
Delete/dqlite/1-workers-4                  4.372 ±  1%         4.365 ± 0%       ~ (p=0.434 n=7)
Delete/dqlite/4-workers-4                  4.360 ±  1%         4.381 ± 0%  +0.48% (p=0.013 n=7)
Delete/dqlite/16-workers-4                 4.366 ±  1%         4.360 ± 0%       ~ (p=0.302 n=7)
Delete/dqlite/64-workers-4                 4.357 ±  0%         4.349 ± 1%       ~ (p=0.738 n=7)
Delete/dqlite/128-workers-4                4.350 ±  1%         4.360 ± 1%       ~ (p=0.383 n=7)
Get/sqlite-4                               0.000 ±  0%         0.000 ± 0%       ~ (p=1.000 n=7) ¹
Get/dqlite-4                               0.000 ±  0%         0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-tiny/all-4                     0.000 ±  0%         0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-tiny/pagination-4              0.000 ±  0%         0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-fits-in-page/all-4             0.000 ±  0%         0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-fits-in-page/pagination-4      0.000 ±  0%         0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-overflows-page/all-4           0.000 ±  0%         0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-overflows-page/pagination-4    0.000 ±  0%         0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-tiny/all-4                     0.000 ±  0%         0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-tiny/pagination-4              0.000 ±  0%         0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-fits-in-page/all-4             0.000 ±  0%         0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-fits-in-page/pagination-4      0.000 ±  0%         0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-overflows-page/all-4           0.000 ±  0%         0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-overflows-page/pagination-4    0.000 ±  0%         0.000 ± 0%       ~ (p=1.000 n=7) ¹
Update/sqlite/1-workers-4                  4.247 ±  0%         4.243 ± 0%       ~ (p=0.124 n=7)
Update/sqlite/4-workers-4                  4.305 ±  0%         4.312 ± 1%       ~ (p=0.535 n=7)
Update/sqlite/16-workers-4                 4.311 ±  0%         4.321 ± 0%       ~ (p=0.433 n=7)
Update/sqlite/64-workers-4                 4.341 ± 23%         4.362 ± 1%       ~ (p=0.710 n=7)
Update/sqlite/128-workers-4                4.444 ±  4%         4.371 ± 6%       ~ (p=0.383 n=7)
Update/dqlite/1-workers-4                  4.205 ±  0%         4.204 ± 0%       ~ (p=0.195 n=7)
Update/dqlite/4-workers-4                  4.219 ±  1%         4.206 ± 0%       ~ (p=0.104 n=7)
Update/dqlite/16-workers-4                 4.223 ±  1%         4.233 ± 2%       ~ (p=0.644 n=7)
Update/dqlite/64-workers-4                 4.308 ±  2%         4.336 ± 2%       ~ (p=0.318 n=7)
Update/dqlite/128-workers-4                4.606 ±  5%         4.517 ± 4%       ~ (p=0.128 n=7)
geomean                                                ²                   -0.03%               ²
¹ all samples are equal
² summaries must be >0 to compute geomean

                                        │     Before     │                After                 │
                                        │ sec-reading/op │ sec-reading/op  vs base              │
Compaction/sqlite-4                        27.40µ ±  38%     27.30µ ± 11%       ~ (p=0.833 n=7)
Compaction/dqlite-4                        31.50µ ±  73%     31.80µ ± 54%       ~ (p=0.875 n=7)
Create/sqlite/1-workers-4                  19.90µ ±  17%     19.00µ ± 17%       ~ (p=0.209 n=7)
Create/sqlite/4-workers-4                  20.50µ ±  19%     18.90µ ± 20%       ~ (p=0.052 n=7)
Create/sqlite/16-workers-4                 21.70µ ±  31%     19.60µ ± 24%       ~ (p=0.644 n=7)
Create/sqlite/64-workers-4                 22.60µ ±  23%     24.00µ ± 75%       ~ (p=0.383 n=7)
Create/sqlite/128-workers-4                22.00µ ±  35%     22.30µ ± 40%       ~ (p=0.902 n=7)
Create/dqlite/1-workers-4                 10.800µ ±  60%     9.400µ ± 67%       ~ (p=0.330 n=7)
Create/dqlite/4-workers-4                  11.30µ ±  60%     10.80µ ± 62%       ~ (p=0.196 n=7)
Create/dqlite/16-workers-4                 11.50µ ±  61%     13.30µ ± 31%       ~ (p=0.535 n=7)
Create/dqlite/64-workers-4                 11.30µ ±  46%     11.60µ ± 38%       ~ (p=0.399 n=7)
Create/dqlite/128-workers-4                11.10µ ±  50%     12.10µ ± 16%       ~ (p=0.456 n=7)
Delete/sqlite/1-workers-4                  31.20µ ±  24%     29.70µ ± 18%       ~ (p=0.218 n=7)
Delete/sqlite/4-workers-4                  33.10µ ±  18%     30.70µ ± 12%       ~ (p=0.318 n=7)
Delete/sqlite/16-workers-4                 33.90µ ±  17%     33.00µ ±  8%       ~ (p=0.512 n=7)
Delete/sqlite/64-workers-4                 34.90µ ±  11%     37.30µ ± 18%       ~ (p=0.436 n=7)
Delete/sqlite/128-workers-4                38.70µ ±  16%     40.00µ ± 24%       ~ (p=0.779 n=7)
Delete/dqlite/1-workers-4                  14.50µ ±  21%     15.10µ ± 16%       ~ (p=0.274 n=7)
Delete/dqlite/4-workers-4                  16.80µ ±  23%     17.70µ ± 26%       ~ (p=0.596 n=7)
Delete/dqlite/16-workers-4                 14.50µ ±  49%     16.60µ ± 46%       ~ (p=0.642 n=7)
Delete/dqlite/64-workers-4                 17.90µ ±  48%     15.00µ ± 53%       ~ (p=0.805 n=7)
Delete/dqlite/128-workers-4                16.00µ ±  44%     13.70µ ± 59%       ~ (p=0.598 n=7)
Get/sqlite-4                               63.90µ ±   5%     65.70µ ±  9%       ~ (p=0.535 n=7)
Get/dqlite-4                               33.70µ ±  13%     33.70µ ± 13%       ~ (p=0.929 n=7)
List/sqlite-tiny/all-4                     6.800µ ±  18%     6.600µ ± 21%       ~ (p=0.780 n=7)
List/sqlite-tiny/pagination-4              14.60µ ±   2%     14.40µ ±  3%       ~ (p=0.076 n=7)
List/sqlite-fits-in-page/all-4             13.30µ ±  23%     11.50µ ± 23%       ~ (p=0.554 n=7)
List/sqlite-fits-in-page/pagination-4      36.80µ ±  12%     33.70µ ± 18%  -8.42% (p=0.010 n=7)
List/sqlite-overflows-page/all-4           13.40µ ±   3%     13.30µ ±  4%       ~ (p=0.883 n=7)
List/sqlite-overflows-page/pagination-4    42.20µ ±  21%     42.50µ ± 25%       ~ (p=0.473 n=7)
List/dqlite-tiny/all-4                     4.100µ ±  22%     4.100µ ± 22%       ~ (p=1.000 n=7)
List/dqlite-tiny/pagination-4              9.400µ ±   7%     9.500µ ± 11%       ~ (p=0.920 n=7)
List/dqlite-fits-in-page/all-4            10.700µ ±  17%     9.200µ ± 21%       ~ (p=0.832 n=7)
List/dqlite-fits-in-page/pagination-4      25.90µ ±  18%     27.00µ ± 16%       ~ (p=0.128 n=7)
List/dqlite-overflows-page/all-4           34.60µ ±  13%     35.20µ ± 13%       ~ (p=0.198 n=7)
List/dqlite-overflows-page/pagination-4    66.60µ ±  18%     68.90µ ± 12%       ~ (p=0.402 n=7)
Update/sqlite/1-workers-4                  20.60µ ±  22%     19.80µ ± 32%       ~ (p=0.736 n=7)
Update/sqlite/4-workers-4                  19.90µ ±  28%     20.60µ ± 44%       ~ (p=0.710 n=7)
Update/sqlite/16-workers-4                 22.00µ ±   7%     21.30µ ± 27%       ~ (p=0.974 n=7)
Update/sqlite/64-workers-4                 21.80µ ± 114%     24.90µ ± 26%       ~ (p=0.383 n=7)
Update/sqlite/128-workers-4                24.20µ ±  24%     23.80µ ± 17%       ~ (p=0.596 n=7)
Update/dqlite/1-workers-4                  13.00µ ±  40%     11.80µ ± 65%       ~ (p=0.830 n=7)
Update/dqlite/4-workers-4                  11.50µ ±  45%     11.10µ ± 45%       ~ (p=0.477 n=7)
Update/dqlite/16-workers-4                 11.60µ ±  28%     10.70µ ± 46%       ~ (p=0.301 n=7)
Update/dqlite/64-workers-4                 11.30µ ±  40%     14.60µ ± 31%       ~ (p=0.104 n=7)
Update/dqlite/128-workers-4                12.60µ ±  28%     12.20µ ± 29%       ~ (p=0.090 n=7)
geomean                                    18.92µ            18.74µ        -0.95%

                                        │     Before      │                  After                  │
                                        │ sec-writing/op  │ sec-writing/op  vs base                 │
Compaction/sqlite-4                       9.100µ ±  20%      9.200µ ±   8%        ~ (p=1.000 n=7)
Compaction/dqlite-4                       6.900µ ±  13%      6.700µ ±  12%        ~ (p=0.976 n=7)
Create/sqlite/1-workers-4                 282.2µ ±   4%      266.8µ ±  13%   -5.46% (p=0.011 n=7)
Create/sqlite/4-workers-4                 1.481m ±   4%      1.525m ±  23%        ~ (p=0.902 n=7)
Create/sqlite/16-workers-4                5.420m ±   5%      4.872m ±  14%        ~ (p=0.259 n=7)
Create/sqlite/64-workers-4                18.46m ±   6%      18.37m ± 195%        ~ (p=0.877 n=7)
Create/sqlite/128-workers-4               35.04m ±  60%      36.41m ±  17%        ~ (p=0.710 n=7)
Create/dqlite/1-workers-4                 39.50µ ±  20%      41.50µ ±  21%        ~ (p=0.513 n=7)
Create/dqlite/4-workers-4                 45.30µ ±  15%      48.90µ ±  19%        ~ (p=0.535 n=7)
Create/dqlite/16-workers-4                43.80µ ±  25%      49.40µ ±  25%        ~ (p=0.209 n=7)
Create/dqlite/64-workers-4                49.60µ ±  19%      46.50µ ±  43%        ~ (p=0.512 n=7)
Create/dqlite/128-workers-4               43.60µ ±  29%      44.70µ ±  29%        ~ (p=0.383 n=7)
Delete/sqlite/1-workers-4                 290.1µ ±  24%      244.8µ ±   9%  -15.62% (p=0.001 n=7)
Delete/sqlite/4-workers-4                 1.466m ±   9%      1.403m ±  10%        ~ (p=0.620 n=7)
Delete/sqlite/16-workers-4                5.558m ±  11%      5.411m ±   8%        ~ (p=0.383 n=7)
Delete/sqlite/64-workers-4                18.83m ±  12%      18.19m ±   8%        ~ (p=0.128 n=7)
Delete/sqlite/128-workers-4               35.30m ±  18%      34.45m ±  14%        ~ (p=0.259 n=7)
Delete/dqlite/1-workers-4                 49.40µ ±  49%      40.50µ ±  31%        ~ (p=0.535 n=7)
Delete/dqlite/4-workers-4                 44.20µ ±  15%      47.60µ ±  19%        ~ (p=0.302 n=7)
Delete/dqlite/16-workers-4                46.10µ ±  19%      46.50µ ±  16%        ~ (p=0.830 n=7)
Delete/dqlite/64-workers-4                47.30µ ±  24%      43.60µ ±  14%        ~ (p=0.535 n=7)
Delete/dqlite/128-workers-4               48.00µ ±  24%      41.90µ ±  38%        ~ (p=0.128 n=7)
Get/sqlite-4                               0.000 ±   0%       0.000 ±   0%        ~ (p=1.000 n=7) ¹
Get/dqlite-4                               0.000 ±   0%       0.000 ±   0%        ~ (p=1.000 n=7) ¹
List/sqlite-tiny/all-4                     0.000 ±   0%       0.000 ±   0%        ~ (p=1.000 n=7) ¹
List/sqlite-tiny/pagination-4              0.000 ±   0%       0.000 ±   0%        ~ (p=1.000 n=7) ¹
List/sqlite-fits-in-page/all-4             0.000 ±   0%       0.000 ±   0%        ~ (p=1.000 n=7) ¹
List/sqlite-fits-in-page/pagination-4      0.000 ±   0%       0.000 ±   0%        ~ (p=1.000 n=7) ¹
List/sqlite-overflows-page/all-4           0.000 ±   0%       0.000 ±   0%        ~ (p=1.000 n=7) ¹
List/sqlite-overflows-page/pagination-4    0.000 ±   0%       0.000 ±   0%        ~ (p=1.000 n=7) ¹
List/dqlite-tiny/all-4                     0.000 ±   0%       0.000 ±   0%        ~ (p=1.000 n=7) ¹
List/dqlite-tiny/pagination-4              0.000 ±   0%       0.000 ±   0%        ~ (p=1.000 n=7) ¹
List/dqlite-fits-in-page/all-4             0.000 ±   0%       0.000 ±   0%        ~ (p=1.000 n=7) ¹
List/dqlite-fits-in-page/pagination-4      0.000 ±   0%       0.000 ±   0%        ~ (p=1.000 n=7) ¹
List/dqlite-overflows-page/all-4           0.000 ±   0%       0.000 ±   0%        ~ (p=1.000 n=7) ¹
List/dqlite-overflows-page/pagination-4    0.000 ±   0%       0.000 ±   0%        ~ (p=1.000 n=7) ¹
Update/sqlite/1-workers-4                 276.0µ ±  18%      283.5µ ±   7%        ~ (p=0.535 n=7)
Update/sqlite/4-workers-4                 1.548m ±  15%      1.422m ±  15%        ~ (p=1.000 n=7)
Update/sqlite/16-workers-4                5.450m ±   9%      5.419m ±   6%        ~ (p=0.805 n=7)
Update/sqlite/64-workers-4                18.86m ± 253%      18.60m ±  10%        ~ (p=0.535 n=7)
Update/sqlite/128-workers-4               38.76m ±  34%      36.49m ±  57%        ~ (p=0.805 n=7)
Update/dqlite/1-workers-4                 39.10µ ±  31%      36.40µ ±  13%   -6.91% (p=0.041 n=7)
Update/dqlite/4-workers-4                 44.50µ ±  14%      48.00µ ±  20%        ~ (p=0.172 n=7)
Update/dqlite/16-workers-4                43.10µ ±  15%      42.00µ ±   8%        ~ (p=0.535 n=7)
Update/dqlite/64-workers-4                43.60µ ±  17%      40.50µ ±  41%        ~ (p=1.000 n=7)
Update/dqlite/128-workers-4               49.10µ ±  22%      52.40µ ±  26%        ~ (p=1.000 n=7)
geomean                                                 ²                    -1.54%               ²
¹ all samples are equal
² summaries must be >0 to compute geomean

                                        │     Before     │                After                │
                                        │      B/op      │     B/op       vs base              │
Compaction/sqlite-4                        2.373Ki ± 47%   2.380Ki ±  9%       ~ (p=1.000 n=7)
Compaction/dqlite-4                        5.392Ki ± 93%   5.707Ki ± 70%       ~ (p=1.000 n=7)
Create/sqlite/1-workers-4                  20.50Ki ±  0%   20.50Ki ±  0%       ~ (p=0.059 n=7)
Create/sqlite/4-workers-4                  20.50Ki ±  0%   20.48Ki ±  0%       ~ (p=0.096 n=7)
Create/sqlite/16-workers-4                 20.48Ki ±  0%   20.48Ki ±  0%       ~ (p=0.778 n=7)
Create/sqlite/64-workers-4                 20.52Ki ±  0%   20.51Ki ±  5%       ~ (p=0.535 n=7)
Create/sqlite/128-workers-4                20.61Ki ±  3%   20.62Ki ±  1%       ~ (p=0.535 n=7)
Create/dqlite/1-workers-4                  19.76Ki ±  0%   19.74Ki ±  0%  -0.10% (p=0.001 n=7)
Create/dqlite/4-workers-4                  19.73Ki ±  0%   19.73Ki ±  0%       ~ (p=0.072 n=7)
Create/dqlite/16-workers-4                 19.73Ki ±  0%   19.72Ki ±  0%       ~ (p=0.222 n=7)
Create/dqlite/64-workers-4                 19.74Ki ±  0%   19.74Ki ±  0%       ~ (p=0.552 n=7)
Create/dqlite/128-workers-4                19.75Ki ±  0%   19.75Ki ±  0%       ~ (p=0.604 n=7)
Delete/sqlite/1-workers-4                  22.94Ki ±  0%   22.92Ki ±  0%  -0.05% (p=0.001 n=7)
Delete/sqlite/4-workers-4                  22.92Ki ±  0%   22.92Ki ±  0%       ~ (p=0.598 n=7)
Delete/sqlite/16-workers-4                 22.90Ki ±  0%   22.91Ki ±  0%       ~ (p=1.000 n=7)
Delete/sqlite/64-workers-4                 22.91Ki ±  0%   22.92Ki ±  1%       ~ (p=0.928 n=7)
Delete/sqlite/128-workers-4                23.02Ki ±  1%   23.05Ki ±  1%       ~ (p=0.535 n=7)
Delete/dqlite/1-workers-4                  22.02Ki ±  0%   22.02Ki ±  0%       ~ (p=0.928 n=7)
Delete/dqlite/4-workers-4                  21.99Ki ±  0%   21.99Ki ±  0%       ~ (p=0.596 n=7)
Delete/dqlite/16-workers-4                 22.00Ki ±  0%   21.99Ki ±  0%       ~ (p=0.924 n=7)
Delete/dqlite/64-workers-4                 22.00Ki ±  0%   22.01Ki ±  0%  +0.04% (p=0.026 n=7)
Delete/dqlite/128-workers-4                22.02Ki ±  0%   22.00Ki ± 10%       ~ (p=0.209 n=7)
Get/sqlite-4                               24.41Ki ±  0%   24.42Ki ±  0%       ~ (p=0.555 n=7)
Get/dqlite-4                               23.25Ki ±  0%   23.25Ki ±  0%       ~ (p=0.167 n=7)
List/sqlite-tiny/all-4                     1.193Ki ± 12%   1.167Ki ± 11%       ~ (p=0.805 n=7)
List/sqlite-tiny/pagination-4              1.579Ki ±  4%   1.559Ki ±  3%  -1.30% (p=0.030 n=7)
List/sqlite-fits-in-page/all-4             7.535Ki ± 19%   6.682Ki ± 15%       ~ (p=0.535 n=7)
List/sqlite-fits-in-page/pagination-4     10.459Ki ± 15%   9.774Ki ± 22%  -6.55% (p=0.026 n=7)
List/sqlite-overflows-page/all-4           20.54Ki ± 12%   20.54Ki ±  0%       ~ (p=0.364 n=7)
List/sqlite-overflows-page/pagination-4    25.70Ki ± 35%   25.09Ki ± 38%       ~ (p=0.456 n=7)
List/dqlite-tiny/all-4                     1.015Ki ±  7%   1.059Ki ±  5%       ~ (p=0.057 n=7)
List/dqlite-tiny/pagination-4                974.0 ±  2%     973.0 ±  4%       ~ (p=0.874 n=7)
List/dqlite-fits-in-page/all-4             6.012Ki ±  9%   5.995Ki ±  4%       ~ (p=1.000 n=7)
List/dqlite-fits-in-page/pagination-4      5.479Ki ±  6%   5.532Ki ±  8%       ~ (p=0.097 n=7)
List/dqlite-overflows-page/all-4           30.08Ki ±  8%   30.17Ki ± 10%       ~ (p=0.805 n=7)
List/dqlite-overflows-page/pagination-4    29.01Ki ±  6%   30.04Ki ±  4%       ~ (p=0.259 n=7)
Update/sqlite/1-workers-4                  22.28Ki ±  0%   22.29Ki ±  0%       ~ (p=0.336 n=7)
Update/sqlite/4-workers-4                  22.30Ki ±  0%   22.29Ki ±  0%       ~ (p=0.120 n=7)
Update/sqlite/16-workers-4                 22.35Ki ±  0%   22.36Ki ±  0%       ~ (p=0.435 n=7)
Update/sqlite/64-workers-4                 22.51Ki ± 34%   22.53Ki ±  1%       ~ (p=0.902 n=7)
Update/sqlite/128-workers-4                23.21Ki ±  4%   23.21Ki ±  4%       ~ (p=0.620 n=7)
Update/dqlite/1-workers-4                  21.49Ki ±  0%   21.49Ki ±  0%       ~ (p=0.066 n=7)
Update/dqlite/4-workers-4                  21.48Ki ±  0%   21.47Ki ±  0%       ~ (p=0.636 n=7)
Update/dqlite/16-workers-4                 21.52Ki ±  1%   21.58Ki ±  1%       ~ (p=0.456 n=7)
Update/dqlite/64-workers-4                 21.88Ki ±  2%   21.90Ki ±  2%       ~ (p=0.456 n=7)
Update/dqlite/128-workers-4                23.19Ki ±  5%   22.92Ki ±  5%       ~ (p=0.165 n=7)
geomean                                    14.35Ki         14.32Ki        -0.24%

                                        │   Before    │                After                 │
                                        │  allocs/op  │  allocs/op   vs base                 │
Compaction/sqlite-4                       51.00 ± 49%   51.00 ±  8%        ~ (p=0.920 n=7)
Compaction/dqlite-4                       129.0 ± 95%   137.0 ± 71%        ~ (p=0.902 n=7)
Create/sqlite/1-workers-4                 399.0 ±  0%   399.0 ±  0%        ~ (p=1.000 n=7) ¹
Create/sqlite/4-workers-4                 399.0 ±  0%   399.0 ±  0%        ~ (p=1.000 n=7) ¹
Create/sqlite/16-workers-4                399.0 ±  0%   399.0 ±  0%        ~ (p=1.000 n=7)
Create/sqlite/64-workers-4                399.0 ±  0%   399.0 ±  2%        ~ (p=0.318 n=7)
Create/sqlite/128-workers-4               400.0 ±  2%   401.0 ±  0%        ~ (p=0.761 n=7)
Create/dqlite/1-workers-4                 369.0 ±  0%   369.0 ±  0%        ~ (p=1.000 n=7)
Create/dqlite/4-workers-4                 369.0 ±  0%   369.0 ±  0%        ~ (p=1.000 n=7)
Create/dqlite/16-workers-4                369.0 ±  0%   369.0 ±  0%        ~ (p=1.000 n=7)
Create/dqlite/64-workers-4                368.0 ±  0%   368.0 ±  0%        ~ (p=1.000 n=7) ¹
Create/dqlite/128-workers-4               368.0 ±  0%   368.0 ±  0%        ~ (p=1.000 n=7) ¹
Delete/sqlite/1-workers-4                 431.0 ±  0%   431.0 ±  0%        ~ (p=1.000 n=7) ¹
Delete/sqlite/4-workers-4                 431.0 ±  0%   431.0 ±  0%        ~ (p=1.000 n=7) ¹
Delete/sqlite/16-workers-4                431.0 ±  0%   431.0 ±  0%        ~ (p=1.000 n=7)
Delete/sqlite/64-workers-4                430.0 ±  0%   430.0 ±  0%        ~ (p=0.773 n=7)
Delete/sqlite/128-workers-4               431.0 ±  0%   432.0 ±  1%        ~ (p=0.388 n=7)
Delete/dqlite/1-workers-4                 399.0 ±  0%   399.0 ±  0%        ~ (p=1.000 n=7) ¹
Delete/dqlite/4-workers-4                 399.0 ±  0%   399.0 ±  0%        ~ (p=1.000 n=7) ¹
Delete/dqlite/16-workers-4                399.0 ±  0%   399.0 ±  0%        ~ (p=1.000 n=7) ¹
Delete/dqlite/64-workers-4                399.0 ±  0%   399.0 ±  0%        ~ (p=1.000 n=7) ¹
Delete/dqlite/128-workers-4               399.0 ±  0%   399.0 ± 13%        ~ (p=1.000 n=7)
Get/sqlite-4                              445.0 ±  0%   445.0 ±  0%        ~ (p=1.000 n=7) ¹
Get/dqlite-4                              412.0 ±  0%   412.0 ±  0%        ~ (p=1.000 n=7) ¹
List/sqlite-tiny/all-4                    16.00 ± 19%   16.00 ± 12%        ~ (p=0.825 n=7)
List/sqlite-tiny/pagination-4             25.00 ±  4%   25.00 ±  4%        ~ (p=0.538 n=7)
List/sqlite-fits-in-page/all-4            21.00 ± 29%   18.00 ± 28%        ~ (p=0.717 n=7)
List/sqlite-fits-in-page/pagination-4     38.00 ± 18%   34.00 ± 29%  -10.53% (p=0.012 n=7)
List/sqlite-overflows-page/all-4          9.000 ±  0%   9.000 ±  0%        ~ (p=1.000 n=7) ¹
List/sqlite-overflows-page/pagination-4   15.00 ± 47%   14.00 ± 57%        ~ (p=0.522 n=7)
List/dqlite-tiny/all-4                    14.00 ±  7%   15.00 ±  7%        ~ (p=0.213 n=7)
List/dqlite-tiny/pagination-4             14.00 ±  7%   14.00 ±  7%        ~ (p=1.000 n=7)
List/dqlite-fits-in-page/all-4            20.00 ±  5%   19.00 ± 11%        ~ (p=0.184 n=7)
List/dqlite-fits-in-page/pagination-4     19.00 ±  5%   20.00 ±  5%        ~ (p=0.429 n=7)
List/dqlite-overflows-page/all-4          33.00 ±  3%   34.00 ±  6%        ~ (p=0.266 n=7)
List/dqlite-overflows-page/pagination-4   33.00 ± 12%   34.00 ±  6%        ~ (p=0.143 n=7)
Update/sqlite/1-workers-4                 412.0 ±  0%   412.0 ±  0%        ~ (p=1.000 n=7) ¹
Update/sqlite/4-workers-4                 412.0 ±  0%   412.0 ±  0%        ~ (p=1.000 n=7)
Update/sqlite/16-workers-4                413.0 ±  0%   413.0 ±  0%        ~ (p=0.853 n=7)
Update/sqlite/64-workers-4                416.0 ± 31%   416.0 ±  1%        ~ (p=0.834 n=7)
Update/sqlite/128-workers-4               428.0 ±  4%   428.0 ±  4%        ~ (p=0.596 n=7)
Update/dqlite/1-workers-4                 381.0 ±  0%   381.0 ±  0%        ~ (p=1.000 n=7) ¹
Update/dqlite/4-workers-4                 381.0 ±  0%   381.0 ±  0%        ~ (p=1.000 n=7)
Update/dqlite/16-workers-4                381.0 ±  1%   383.0 ±  1%        ~ (p=0.330 n=7)
Update/dqlite/64-workers-4                388.0 ±  2%   388.0 ±  2%        ~ (p=0.479 n=7)
Update/dqlite/128-workers-4               411.0 ±  5%   405.0 ±  5%        ~ (p=0.194 n=7)
geomean                                   170.7         170.1         -0.33%
¹ all samples are equal

                                        │        Before         │                    After                    │
                                        │ network-bytes-read/op │ network-bytes-read/op  vs base              │
Compaction/dqlite-4                               1.949Ki ± 95%           2.066Ki ± 72%       ~ (p=0.976 n=7)
Create/dqlite/1-workers-4                           240.6 ±  0%             241.4 ±  0%  +0.33% (p=0.001 n=7)
Create/dqlite/4-workers-4                           241.5 ±  0%             241.8 ±  0%       ~ (p=0.131 n=7)
Create/dqlite/16-workers-4                          241.7 ±  0%             242.1 ±  0%       ~ (p=0.136 n=7)
Create/dqlite/64-workers-4                          242.0 ±  0%             241.8 ±  0%       ~ (p=0.979 n=7)
Create/dqlite/128-workers-4                         241.9 ±  1%             242.2 ±  0%  +0.12% (p=0.034 n=7)
Delete/dqlite/1-workers-4                           717.6 ±  0%             717.4 ±  0%       ~ (p=0.902 n=7)
Delete/dqlite/4-workers-4                           718.9 ±  0%             719.0 ±  0%       ~ (p=0.780 n=7)
Delete/dqlite/16-workers-4                          719.0 ±  0%             718.8 ±  0%       ~ (p=0.540 n=7)
Delete/dqlite/64-workers-4                          719.1 ±  0%             718.7 ±  0%       ~ (p=0.514 n=7)
Delete/dqlite/128-workers-4                         719.4 ±  0%             719.5 ± 12%       ~ (p=0.830 n=7)
Get/dqlite-4                                      1.133Ki ±  0%           1.133Ki ±  0%       ~ (p=0.122 n=7)
List/dqlite-tiny/all-4                              210.3 ±  8%             213.3 ±  2%  +1.43% (p=0.029 n=7)
List/dqlite-tiny/pagination-4                       197.3 ±  2%             197.7 ±  1%       ~ (p=1.000 n=7)
List/dqlite-fits-in-page/all-4                    1.360Ki ±  5%           1.321Ki ±  9%       ~ (p=0.221 n=7)
List/dqlite-fits-in-page/pagination-4             1.318Ki ±  7%           1.396Ki ±  5%       ~ (p=0.051 n=7)
List/dqlite-overflows-page/all-4                  6.925Ki ±  1%           7.006Ki ±  4%       ~ (p=0.209 n=7)
List/dqlite-overflows-page/pagination-4           6.919Ki ± 14%           7.094Ki ±  6%       ~ (p=0.209 n=7)
Update/dqlite/1-workers-4                           262.9 ±  0%             262.8 ±  0%       ~ (p=0.140 n=7)
Update/dqlite/4-workers-4                           263.1 ±  0%             263.2 ±  0%       ~ (p=0.831 n=7)
Update/dqlite/16-workers-4                          263.1 ±  1%             264.0 ±  1%       ~ (p=0.462 n=7)
Update/dqlite/64-workers-4                          267.4 ±  2%             267.7 ±  2%       ~ (p=0.439 n=7)
Update/dqlite/128-workers-4                         283.6 ±  5%             279.3 ±  5%       ~ (p=0.209 n=7)
geomean                                             564.9                   568.2        +0.58%

                                        │          Before          │                     After                      │
                                        │ network-bytes-written/op │ network-bytes-written/op  vs base              │
Compaction/dqlite-4                                    4.561 ± 15%                4.597 ± 13%       ~ (p=0.780 n=7)
Create/dqlite/1-workers-4                              114.4 ±  1%                115.8 ±  0%  +1.22% (p=0.001 n=7)
Create/dqlite/4-workers-4                              116.3 ±  1%                116.6 ±  0%       ~ (p=0.201 n=7)
Create/dqlite/16-workers-4                             116.3 ±  1%                117.0 ±  1%       ~ (p=0.318 n=7)
Create/dqlite/64-workers-4                             116.9 ±  1%                116.8 ±  1%       ~ (p=0.730 n=7)
Create/dqlite/128-workers-4                            116.9 ±  2%                117.5 ±  1%  +0.51% (p=0.041 n=7)
Delete/dqlite/1-workers-4                              82.38 ±  1%                82.26 ±  1%       ~ (p=1.000 n=7)
Delete/dqlite/4-workers-4                              82.95 ±  0%                83.00 ±  1%       ~ (p=0.620 n=7)
Delete/dqlite/16-workers-4                             82.91 ±  0%                82.75 ±  1%       ~ (p=0.456 n=7)
Delete/dqlite/64-workers-4                             82.90 ±  0%                82.88 ±  0%       ~ (p=0.474 n=7)
Delete/dqlite/128-workers-4                            83.12 ±  1%                83.21 ± 33%       ~ (p=0.778 n=7)
Get/dqlite-4                                           80.50 ±  0%                80.56 ±  0%       ~ (p=0.139 n=7)
List/dqlite-tiny/all-4                               0.04589 ± 15%              0.04715 ±  3%  +2.75% (p=0.017 n=7)
List/dqlite-tiny/pagination-4                        0.05020 ±  6%              0.04871 ±  9%       ~ (p=0.710 n=7)
List/dqlite-fits-in-page/all-4                       0.06210 ±  8%              0.05951 ± 15%       ~ (p=0.318 n=7)
List/dqlite-fits-in-page/pagination-4                0.08400 ± 13%              0.08819 ± 13%       ~ (p=0.620 n=7)
List/dqlite-overflows-page/all-4                     0.07718 ±  3%              0.07914 ±  6%       ~ (p=0.456 n=7)
List/dqlite-overflows-page/pagination-4               0.1152 ± 21%               0.1228 ± 32%       ~ (p=0.383 n=7)
Update/dqlite/1-workers-4                              136.7 ±  0%                136.8 ±  0%       ~ (p=0.298 n=7)
Update/dqlite/4-workers-4                              136.7 ±  0%                136.7 ±  0%       ~ (p=0.615 n=7)
Update/dqlite/16-workers-4                             137.0 ±  1%                137.1 ±  1%       ~ (p=1.000 n=7)
Update/dqlite/64-workers-4                             139.4 ±  2%                139.7 ±  2%       ~ (p=0.383 n=7)
Update/dqlite/128-workers-4                            147.6 ±  5%                146.1 ±  6%       ~ (p=0.259 n=7)
geomean                                                13.82                      13.89        +0.51%
Current status
goos: linux
goarch: amd64
pkg: github.com/canonical/k8s-dqlite/test
cpu: AMD EPYC 7763 64-Core Processor                
                                        │     After      │
                                        │     sec/op     │
Compaction/sqlite-4                       19.70µ ±    8%
Compaction/dqlite-4                       41.07µ ±    5%
Create/sqlite/1-workers-4                 484.7µ ±    8%
Create/sqlite/4-workers-4                 459.6µ ±   10%
Create/sqlite/16-workers-4                430.8µ ±   19%
Create/sqlite/64-workers-4                461.5µ ± 1060%
Create/sqlite/128-workers-4               737.5µ ±   48%
Create/dqlite/1-workers-4                 1.055m ±    3%
Create/dqlite/4-workers-4                 968.5µ ±    2%
Create/dqlite/16-workers-4                952.8µ ±    1%
Create/dqlite/64-workers-4                965.3µ ±    1%
Create/dqlite/128-workers-4               962.2µ ±    6%
Delete/sqlite/1-workers-4                 461.2µ ±    4%
Delete/sqlite/4-workers-4                 453.6µ ±    4%
Delete/sqlite/16-workers-4                439.0µ ±    5%
Delete/sqlite/64-workers-4                447.5µ ±   72%
Delete/sqlite/128-workers-4               612.7µ ±   73%
Delete/dqlite/1-workers-4                 1.061m ±    6%
Delete/dqlite/4-workers-4                 961.3µ ±    3%
Delete/dqlite/16-workers-4                967.2µ ±    3%
Delete/dqlite/64-workers-4                949.3µ ±    3%
Delete/dqlite/128-workers-4               918.4µ ±   12%
Get/sqlite-4                              232.3µ ±    1%
Get/dqlite-4                              354.1µ ±    1%
List/sqlite-tiny/all-4                    3.932µ ±    7%
List/sqlite-tiny/pagination-4             8.149µ ±    2%
List/sqlite-fits-in-page/all-4            7.585µ ±    4%
List/sqlite-fits-in-page/pagination-4     21.29µ ±    4%
List/sqlite-overflows-page/all-4          17.00µ ±    3%
List/sqlite-overflows-page/pagination-4   42.92µ ±    6%
List/dqlite-tiny/all-4                    2.975µ ±    2%
List/dqlite-tiny/pagination-4             6.065µ ±    1%
List/dqlite-fits-in-page/all-4            6.634µ ±    6%
List/dqlite-fits-in-page/pagination-4     17.40µ ±    2%
List/dqlite-overflows-page/all-4          21.32µ ±    2%
List/dqlite-overflows-page/pagination-4   45.09µ ±    5%
Update/sqlite/1-workers-4                 510.5µ ±    3%
Update/sqlite/4-workers-4                 479.9µ ±   19%
Update/sqlite/16-workers-4                453.8µ ±    8%
Update/sqlite/64-workers-4                497.4µ ±   40%
Update/sqlite/128-workers-4               725.5µ ±  175%
Update/dqlite/1-workers-4                 1.077m ±    5%
Update/dqlite/4-workers-4                 950.7µ ±    3%
Update/dqlite/16-workers-4                950.8µ ±    6%
Update/dqlite/64-workers-4                978.5µ ±    9%
Update/dqlite/128-workers-4               965.1µ ±   12%
geomean                                   202.3µ

                                        │        After         │
                                        │ page-cache-misses/op │
Compaction/sqlite-4                               0.000 ± 0%
Compaction/dqlite-4                               0.000 ± 0%
Create/sqlite/1-workers-4                         0.000 ± 0%
Create/sqlite/4-workers-4                         0.000 ± 0%
Create/sqlite/16-workers-4                        0.000 ± 0%
Create/sqlite/64-workers-4                        0.000 ± 0%
Create/sqlite/128-workers-4                       0.000 ± 0%
Create/dqlite/1-workers-4                         0.000 ± 0%
Create/dqlite/4-workers-4                         0.000 ± 0%
Create/dqlite/16-workers-4                        0.000 ± 0%
Create/dqlite/64-workers-4                        0.000 ± 0%
Create/dqlite/128-workers-4                       0.000 ± 0%
Delete/sqlite/1-workers-4                         0.000 ± 0%
Delete/sqlite/4-workers-4                         0.000 ± 0%
Delete/sqlite/16-workers-4                        0.000 ± 0%
Delete/sqlite/64-workers-4                        0.000 ± 0%
Delete/sqlite/128-workers-4                       0.000 ± 0%
Delete/dqlite/1-workers-4                         0.000 ± 0%
Delete/dqlite/4-workers-4                         0.000 ± 0%
Delete/dqlite/16-workers-4                        0.000 ± 0%
Delete/dqlite/64-workers-4                        0.000 ± 0%
Delete/dqlite/128-workers-4                       0.000 ± 0%
Get/sqlite-4                                      0.000 ± 0%
Get/dqlite-4                                      0.000 ± 0%
List/sqlite-tiny/all-4                            0.000 ± 0%
List/sqlite-tiny/pagination-4                     0.000 ± 0%
List/sqlite-fits-in-page/all-4                    0.000 ± 0%
List/sqlite-fits-in-page/pagination-4             0.000 ± 0%
List/sqlite-overflows-page/all-4                  0.000 ± 0%
List/sqlite-overflows-page/pagination-4           0.000 ± 0%
List/dqlite-tiny/all-4                            0.000 ± 0%
List/dqlite-tiny/pagination-4                     0.000 ± 0%
List/dqlite-fits-in-page/all-4                    0.000 ± 0%
List/dqlite-fits-in-page/pagination-4             0.000 ± 0%
List/dqlite-overflows-page/all-4                  0.000 ± 0%
List/dqlite-overflows-page/pagination-4           0.000 ± 0%
Update/sqlite/1-workers-4                         0.000 ± 0%
Update/sqlite/4-workers-4                         0.000 ± 0%
Update/sqlite/16-workers-4                        0.000 ± 0%
Update/sqlite/64-workers-4                        0.000 ± 0%
Update/sqlite/128-workers-4                       0.000 ± 0%
Update/dqlite/1-workers-4                         0.000 ± 0%
Update/dqlite/4-workers-4                         0.000 ± 0%
Update/dqlite/16-workers-4                        0.000 ± 0%
Update/dqlite/64-workers-4                        0.000 ± 0%
Update/dqlite/128-workers-4                       0.000 ± 0%
geomean                                                      ¹
¹ summaries must be >0 to compute geomean

                                        │        After         │
                                        │ page-cache-spills/op │
Compaction/sqlite-4                               0.000 ± 0%
Compaction/dqlite-4                               0.000 ± 0%
Create/sqlite/1-workers-4                         0.000 ± 0%
Create/sqlite/4-workers-4                         0.000 ± 0%
Create/sqlite/16-workers-4                        0.000 ± 0%
Create/sqlite/64-workers-4                        0.000 ± 0%
Create/sqlite/128-workers-4                       0.000 ± 0%
Create/dqlite/1-workers-4                         0.000 ± 0%
Create/dqlite/4-workers-4                         0.000 ± 0%
Create/dqlite/16-workers-4                        0.000 ± 0%
Create/dqlite/64-workers-4                        0.000 ± 0%
Create/dqlite/128-workers-4                       0.000 ± 0%
Delete/sqlite/1-workers-4                         0.000 ± 0%
Delete/sqlite/4-workers-4                         0.000 ± 0%
Delete/sqlite/16-workers-4                        0.000 ± 0%
Delete/sqlite/64-workers-4                        0.000 ± 0%
Delete/sqlite/128-workers-4                       0.000 ± 0%
Delete/dqlite/1-workers-4                         0.000 ± 0%
Delete/dqlite/4-workers-4                         0.000 ± 0%
Delete/dqlite/16-workers-4                        0.000 ± 0%
Delete/dqlite/64-workers-4                        0.000 ± 0%
Delete/dqlite/128-workers-4                       0.000 ± 0%
Get/sqlite-4                                      0.000 ± 0%
Get/dqlite-4                                      0.000 ± 0%
List/sqlite-tiny/all-4                            0.000 ± 0%
List/sqlite-tiny/pagination-4                     0.000 ± 0%
List/sqlite-fits-in-page/all-4                    0.000 ± 0%
List/sqlite-fits-in-page/pagination-4             0.000 ± 0%
List/sqlite-overflows-page/all-4                  0.000 ± 0%
List/sqlite-overflows-page/pagination-4           0.000 ± 0%
List/dqlite-tiny/all-4                            0.000 ± 0%
List/dqlite-tiny/pagination-4                     0.000 ± 0%
List/dqlite-fits-in-page/all-4                    0.000 ± 0%
List/dqlite-fits-in-page/pagination-4             0.000 ± 0%
List/dqlite-overflows-page/all-4                  0.000 ± 0%
List/dqlite-overflows-page/pagination-4           0.000 ± 0%
Update/sqlite/1-workers-4                         0.000 ± 0%
Update/sqlite/4-workers-4                         0.000 ± 0%
Update/sqlite/16-workers-4                        0.000 ± 0%
Update/sqlite/64-workers-4                        0.000 ± 0%
Update/sqlite/128-workers-4                       0.000 ± 0%
Update/dqlite/1-workers-4                         0.000 ± 0%
Update/dqlite/4-workers-4                         0.000 ± 0%
Update/dqlite/16-workers-4                        0.000 ± 0%
Update/dqlite/64-workers-4                        0.000 ± 0%
Update/dqlite/128-workers-4                       0.000 ± 0%
geomean                                                      ¹
¹ summaries must be >0 to compute geomean

                                        │     After     │
                                        │ page-reads/op │
Compaction/sqlite-4                         12.22 ±  1%
Compaction/dqlite-4                         12.17 ±  1%
Create/sqlite/1-workers-4                   13.88 ±  0%
Create/sqlite/4-workers-4                   12.79 ±  3%
Create/sqlite/16-workers-4                  10.78 ±  9%
Create/sqlite/64-workers-4                  8.767 ± 32%
Create/sqlite/128-workers-4                 8.294 ±  6%
Create/dqlite/1-workers-4                   13.33 ±  1%
Create/dqlite/4-workers-4                   10.89 ±  1%
Create/dqlite/16-workers-4                  10.91 ±  1%
Create/dqlite/64-workers-4                  10.91 ±  1%
Create/dqlite/128-workers-4                 10.95 ±  1%
Delete/sqlite/1-workers-4                   16.23 ±  0%
Delete/sqlite/4-workers-4                   14.75 ±  4%
Delete/sqlite/16-workers-4                  12.34 ±  5%
Delete/sqlite/64-workers-4                  10.01 ±  6%
Delete/sqlite/128-workers-4                 9.327 ±  2%
Delete/dqlite/1-workers-4                   15.56 ±  1%
Delete/dqlite/4-workers-4                   13.38 ±  1%
Delete/dqlite/16-workers-4                  13.32 ±  1%
Delete/dqlite/64-workers-4                  13.18 ±  1%
Delete/dqlite/128-workers-4                 13.10 ±  2%
Get/sqlite-4                                14.95 ±  0%
Get/dqlite-4                                11.95 ±  0%
List/sqlite-tiny/all-4                     191.3m ±  1%
List/sqlite-tiny/pagination-4               1.131 ±  0%
List/sqlite-fits-in-page/all-4             162.4m ±  2%
List/sqlite-fits-in-page/pagination-4      962.3m ±  1%
List/sqlite-overflows-page/all-4           177.1m ±  0%
List/sqlite-overflows-page/pagination-4     1.399 ± 30%
List/dqlite-tiny/all-4                     191.5m ±  0%
List/dqlite-tiny/pagination-4               1.126 ±  3%
List/dqlite-fits-in-page/all-4             163.8m ±  1%
List/dqlite-fits-in-page/pagination-4      955.2m ±  3%
List/dqlite-overflows-page/all-4           184.0m ±  0%
List/dqlite-overflows-page/pagination-4     1.071 ±  3%
Update/sqlite/1-workers-4                   15.81 ±  0%
Update/sqlite/4-workers-4                   14.77 ±  3%
Update/sqlite/16-workers-4                  12.62 ±  4%
Update/sqlite/64-workers-4                  11.11 ±  7%
Update/sqlite/128-workers-4                 10.62 ±  8%
Update/dqlite/1-workers-4                   15.42 ±  1%
Update/dqlite/4-workers-4                   12.78 ±  1%
Update/dqlite/16-workers-4                  12.76 ±  1%
Update/dqlite/64-workers-4                  12.95 ±  3%
Update/dqlite/128-workers-4                 13.47 ±  4%
geomean                                     5.189

                                        │     After      │
                                        │ page-writes/op │
Compaction/sqlite-4                        190.5m ± 0%
Compaction/dqlite-4                        187.4m ± 0%
Create/sqlite/1-workers-4                   4.360 ± 0%
Create/sqlite/4-workers-4                   4.326 ± 1%
Create/sqlite/16-workers-4                  4.298 ± 1%
Create/sqlite/64-workers-4                  4.246 ± 5%
Create/sqlite/128-workers-4                 4.210 ± 2%
Create/dqlite/1-workers-4                   4.266 ± 1%
Create/dqlite/4-workers-4                   4.286 ± 0%
Create/dqlite/16-workers-4                  4.297 ± 0%
Create/dqlite/64-workers-4                  4.288 ± 0%
Create/dqlite/128-workers-4                 4.300 ± 0%
Delete/sqlite/1-workers-4                   4.390 ± 0%
Delete/sqlite/4-workers-4                   4.378 ± 1%
Delete/sqlite/16-workers-4                  4.359 ± 1%
Delete/sqlite/64-workers-4                  4.360 ± 1%
Delete/sqlite/128-workers-4                 4.348 ± 1%
Delete/dqlite/1-workers-4                   4.365 ± 0%
Delete/dqlite/4-workers-4                   4.381 ± 0%
Delete/dqlite/16-workers-4                  4.360 ± 0%
Delete/dqlite/64-workers-4                  4.349 ± 1%
Delete/dqlite/128-workers-4                 4.360 ± 1%
Get/sqlite-4                                0.000 ± 0%
Get/dqlite-4                                0.000 ± 0%
List/sqlite-tiny/all-4                      0.000 ± 0%
List/sqlite-tiny/pagination-4               0.000 ± 0%
List/sqlite-fits-in-page/all-4              0.000 ± 0%
List/sqlite-fits-in-page/pagination-4       0.000 ± 0%
List/sqlite-overflows-page/all-4            0.000 ± 0%
List/sqlite-overflows-page/pagination-4     0.000 ± 0%
List/dqlite-tiny/all-4                      0.000 ± 0%
List/dqlite-tiny/pagination-4               0.000 ± 0%
List/dqlite-fits-in-page/all-4              0.000 ± 0%
List/dqlite-fits-in-page/pagination-4       0.000 ± 0%
List/dqlite-overflows-page/all-4            0.000 ± 0%
List/dqlite-overflows-page/pagination-4     0.000 ± 0%
Update/sqlite/1-workers-4                   4.243 ± 0%
Update/sqlite/4-workers-4                   4.312 ± 1%
Update/sqlite/16-workers-4                  4.321 ± 0%
Update/sqlite/64-workers-4                  4.362 ± 1%
Update/sqlite/128-workers-4                 4.371 ± 6%
Update/dqlite/1-workers-4                   4.204 ± 0%
Update/dqlite/4-workers-4                   4.206 ± 0%
Update/dqlite/16-workers-4                  4.233 ± 2%
Update/dqlite/64-workers-4                  4.336 ± 2%
Update/dqlite/128-workers-4                 4.517 ± 4%
geomean                                                ¹
¹ summaries must be >0 to compute geomean

                                        │     After      │
                                        │ sec-reading/op │
Compaction/sqlite-4                         27.30µ ± 11%
Compaction/dqlite-4                         31.80µ ± 54%
Create/sqlite/1-workers-4                   19.00µ ± 17%
Create/sqlite/4-workers-4                   18.90µ ± 20%
Create/sqlite/16-workers-4                  19.60µ ± 24%
Create/sqlite/64-workers-4                  24.00µ ± 75%
Create/sqlite/128-workers-4                 22.30µ ± 40%
Create/dqlite/1-workers-4                   9.400µ ± 67%
Create/dqlite/4-workers-4                   10.80µ ± 62%
Create/dqlite/16-workers-4                  13.30µ ± 31%
Create/dqlite/64-workers-4                  11.60µ ± 38%
Create/dqlite/128-workers-4                 12.10µ ± 16%
Delete/sqlite/1-workers-4                   29.70µ ± 18%
Delete/sqlite/4-workers-4                   30.70µ ± 12%
Delete/sqlite/16-workers-4                  33.00µ ±  8%
Delete/sqlite/64-workers-4                  37.30µ ± 18%
Delete/sqlite/128-workers-4                 40.00µ ± 24%
Delete/dqlite/1-workers-4                   15.10µ ± 16%
Delete/dqlite/4-workers-4                   17.70µ ± 26%
Delete/dqlite/16-workers-4                  16.60µ ± 46%
Delete/dqlite/64-workers-4                  15.00µ ± 53%
Delete/dqlite/128-workers-4                 13.70µ ± 59%
Get/sqlite-4                                65.70µ ±  9%
Get/dqlite-4                                33.70µ ± 13%
List/sqlite-tiny/all-4                      6.600µ ± 21%
List/sqlite-tiny/pagination-4               14.40µ ±  3%
List/sqlite-fits-in-page/all-4              11.50µ ± 23%
List/sqlite-fits-in-page/pagination-4       33.70µ ± 18%
List/sqlite-overflows-page/all-4            13.30µ ±  4%
List/sqlite-overflows-page/pagination-4     42.50µ ± 25%
List/dqlite-tiny/all-4                      4.100µ ± 22%
List/dqlite-tiny/pagination-4               9.500µ ± 11%
List/dqlite-fits-in-page/all-4              9.200µ ± 21%
List/dqlite-fits-in-page/pagination-4       27.00µ ± 16%
List/dqlite-overflows-page/all-4            35.20µ ± 13%
List/dqlite-overflows-page/pagination-4     68.90µ ± 12%
Update/sqlite/1-workers-4                   19.80µ ± 32%
Update/sqlite/4-workers-4                   20.60µ ± 44%
Update/sqlite/16-workers-4                  21.30µ ± 27%
Update/sqlite/64-workers-4                  24.90µ ± 26%
Update/sqlite/128-workers-4                 23.80µ ± 17%
Update/dqlite/1-workers-4                   11.80µ ± 65%
Update/dqlite/4-workers-4                   11.10µ ± 45%
Update/dqlite/16-workers-4                  10.70µ ± 46%
Update/dqlite/64-workers-4                  14.60µ ± 31%
Update/dqlite/128-workers-4                 12.20µ ± 29%
geomean                                     18.74µ

                                        │      After      │
                                        │ sec-writing/op  │
Compaction/sqlite-4                       9.200µ ±   8%
Compaction/dqlite-4                       6.700µ ±  12%
Create/sqlite/1-workers-4                 266.8µ ±  13%
Create/sqlite/4-workers-4                 1.525m ±  23%
Create/sqlite/16-workers-4                4.872m ±  14%
Create/sqlite/64-workers-4                18.37m ± 195%
Create/sqlite/128-workers-4               36.41m ±  17%
Create/dqlite/1-workers-4                 41.50µ ±  21%
Create/dqlite/4-workers-4                 48.90µ ±  19%
Create/dqlite/16-workers-4                49.40µ ±  25%
Create/dqlite/64-workers-4                46.50µ ±  43%
Create/dqlite/128-workers-4               44.70µ ±  29%
Delete/sqlite/1-workers-4                 244.8µ ±   9%
Delete/sqlite/4-workers-4                 1.403m ±  10%
Delete/sqlite/16-workers-4                5.411m ±   8%
Delete/sqlite/64-workers-4                18.19m ±   8%
Delete/sqlite/128-workers-4               34.45m ±  14%
Delete/dqlite/1-workers-4                 40.50µ ±  31%
Delete/dqlite/4-workers-4                 47.60µ ±  19%
Delete/dqlite/16-workers-4                46.50µ ±  16%
Delete/dqlite/64-workers-4                43.60µ ±  14%
Delete/dqlite/128-workers-4               41.90µ ±  38%
Get/sqlite-4                               0.000 ±   0%
Get/dqlite-4                               0.000 ±   0%
List/sqlite-tiny/all-4                     0.000 ±   0%
List/sqlite-tiny/pagination-4              0.000 ±   0%
List/sqlite-fits-in-page/all-4             0.000 ±   0%
List/sqlite-fits-in-page/pagination-4      0.000 ±   0%
List/sqlite-overflows-page/all-4           0.000 ±   0%
List/sqlite-overflows-page/pagination-4    0.000 ±   0%
List/dqlite-tiny/all-4                     0.000 ±   0%
List/dqlite-tiny/pagination-4              0.000 ±   0%
List/dqlite-fits-in-page/all-4             0.000 ±   0%
List/dqlite-fits-in-page/pagination-4      0.000 ±   0%
List/dqlite-overflows-page/all-4           0.000 ±   0%
List/dqlite-overflows-page/pagination-4    0.000 ±   0%
Update/sqlite/1-workers-4                 283.5µ ±   7%
Update/sqlite/4-workers-4                 1.422m ±  15%
Update/sqlite/16-workers-4                5.419m ±   6%
Update/sqlite/64-workers-4                18.60m ±  10%
Update/sqlite/128-workers-4               36.49m ±  57%
Update/dqlite/1-workers-4                 36.40µ ±  13%
Update/dqlite/4-workers-4                 48.00µ ±  20%
Update/dqlite/16-workers-4                42.00µ ±   8%
Update/dqlite/64-workers-4                40.50µ ±  41%
Update/dqlite/128-workers-4               52.40µ ±  26%
geomean                                                 ¹
¹ summaries must be >0 to compute geomean

                                        │     After     │
                                        │     B/op      │
Compaction/sqlite-4                       2.380Ki ±  9%
Compaction/dqlite-4                       5.707Ki ± 70%
Create/sqlite/1-workers-4                 20.50Ki ±  0%
Create/sqlite/4-workers-4                 20.48Ki ±  0%
Create/sqlite/16-workers-4                20.48Ki ±  0%
Create/sqlite/64-workers-4                20.51Ki ±  5%
Create/sqlite/128-workers-4               20.62Ki ±  1%
Create/dqlite/1-workers-4                 19.74Ki ±  0%
Create/dqlite/4-workers-4                 19.73Ki ±  0%
Create/dqlite/16-workers-4                19.72Ki ±  0%
Create/dqlite/64-workers-4                19.74Ki ±  0%
Create/dqlite/128-workers-4               19.75Ki ±  0%
Delete/sqlite/1-workers-4                 22.92Ki ±  0%
Delete/sqlite/4-workers-4                 22.92Ki ±  0%
Delete/sqlite/16-workers-4                22.91Ki ±  0%
Delete/sqlite/64-workers-4                22.92Ki ±  1%
Delete/sqlite/128-workers-4               23.05Ki ±  1%
Delete/dqlite/1-workers-4                 22.02Ki ±  0%
Delete/dqlite/4-workers-4                 21.99Ki ±  0%
Delete/dqlite/16-workers-4                21.99Ki ±  0%
Delete/dqlite/64-workers-4                22.01Ki ±  0%
Delete/dqlite/128-workers-4               22.00Ki ± 10%
Get/sqlite-4                              24.42Ki ±  0%
Get/dqlite-4                              23.25Ki ±  0%
List/sqlite-tiny/all-4                    1.167Ki ± 11%
List/sqlite-tiny/pagination-4             1.559Ki ±  3%
List/sqlite-fits-in-page/all-4            6.682Ki ± 15%
List/sqlite-fits-in-page/pagination-4     9.774Ki ± 22%
List/sqlite-overflows-page/all-4          20.54Ki ±  0%
List/sqlite-overflows-page/pagination-4   25.09Ki ± 38%
List/dqlite-tiny/all-4                    1.059Ki ±  5%
List/dqlite-tiny/pagination-4               973.0 ±  4%
List/dqlite-fits-in-page/all-4            5.995Ki ±  4%
List/dqlite-fits-in-page/pagination-4     5.532Ki ±  8%
List/dqlite-overflows-page/all-4          30.17Ki ± 10%
List/dqlite-overflows-page/pagination-4   30.04Ki ±  4%
Update/sqlite/1-workers-4                 22.29Ki ±  0%
Update/sqlite/4-workers-4                 22.29Ki ±  0%
Update/sqlite/16-workers-4                22.36Ki ±  0%
Update/sqlite/64-workers-4                22.53Ki ±  1%
Update/sqlite/128-workers-4               23.21Ki ±  4%
Update/dqlite/1-workers-4                 21.49Ki ±  0%
Update/dqlite/4-workers-4                 21.47Ki ±  0%
Update/dqlite/16-workers-4                21.58Ki ±  1%
Update/dqlite/64-workers-4                21.90Ki ±  2%
Update/dqlite/128-workers-4               22.92Ki ±  5%
geomean                                   14.32Ki

                                        │    After    │
                                        │  allocs/op  │
Compaction/sqlite-4                       51.00 ±  8%
Compaction/dqlite-4                       137.0 ± 71%
Create/sqlite/1-workers-4                 399.0 ±  0%
Create/sqlite/4-workers-4                 399.0 ±  0%
Create/sqlite/16-workers-4                399.0 ±  0%
Create/sqlite/64-workers-4                399.0 ±  2%
Create/sqlite/128-workers-4               401.0 ±  0%
Create/dqlite/1-workers-4                 369.0 ±  0%
Create/dqlite/4-workers-4                 369.0 ±  0%
Create/dqlite/16-workers-4                369.0 ±  0%
Create/dqlite/64-workers-4                368.0 ±  0%
Create/dqlite/128-workers-4               368.0 ±  0%
Delete/sqlite/1-workers-4                 431.0 ±  0%
Delete/sqlite/4-workers-4                 431.0 ±  0%
Delete/sqlite/16-workers-4                431.0 ±  0%
Delete/sqlite/64-workers-4                430.0 ±  0%
Delete/sqlite/128-workers-4               432.0 ±  1%
Delete/dqlite/1-workers-4                 399.0 ±  0%
Delete/dqlite/4-workers-4                 399.0 ±  0%
Delete/dqlite/16-workers-4                399.0 ±  0%
Delete/dqlite/64-workers-4                399.0 ±  0%
Delete/dqlite/128-workers-4               399.0 ± 13%
Get/sqlite-4                              445.0 ±  0%
Get/dqlite-4                              412.0 ±  0%
List/sqlite-tiny/all-4                    16.00 ± 12%
List/sqlite-tiny/pagination-4             25.00 ±  4%
List/sqlite-fits-in-page/all-4            18.00 ± 28%
List/sqlite-fits-in-page/pagination-4     34.00 ± 29%
List/sqlite-overflows-page/all-4          9.000 ±  0%
List/sqlite-overflows-page/pagination-4   14.00 ± 57%
List/dqlite-tiny/all-4                    15.00 ±  7%
List/dqlite-tiny/pagination-4             14.00 ±  7%
List/dqlite-fits-in-page/all-4            19.00 ± 11%
List/dqlite-fits-in-page/pagination-4     20.00 ±  5%
List/dqlite-overflows-page/all-4          34.00 ±  6%
List/dqlite-overflows-page/pagination-4   34.00 ±  6%
Update/sqlite/1-workers-4                 412.0 ±  0%
Update/sqlite/4-workers-4                 412.0 ±  0%
Update/sqlite/16-workers-4                413.0 ±  0%
Update/sqlite/64-workers-4                416.0 ±  1%
Update/sqlite/128-workers-4               428.0 ±  4%
Update/dqlite/1-workers-4                 381.0 ±  0%
Update/dqlite/4-workers-4                 381.0 ±  0%
Update/dqlite/16-workers-4                383.0 ±  1%
Update/dqlite/64-workers-4                388.0 ±  2%
Update/dqlite/128-workers-4               405.0 ±  5%
geomean                                   170.1

                                        │         After         │
                                        │ network-bytes-read/op │
Compaction/dqlite-4                               2.066Ki ± 72%
Create/dqlite/1-workers-4                           241.4 ±  0%
Create/dqlite/4-workers-4                           241.8 ±  0%
Create/dqlite/16-workers-4                          242.1 ±  0%
Create/dqlite/64-workers-4                          241.8 ±  0%
Create/dqlite/128-workers-4                         242.2 ±  0%
Delete/dqlite/1-workers-4                           717.4 ±  0%
Delete/dqlite/4-workers-4                           719.0 ±  0%
Delete/dqlite/16-workers-4                          718.8 ±  0%
Delete/dqlite/64-workers-4                          718.7 ±  0%
Delete/dqlite/128-workers-4                         719.5 ± 12%
Get/dqlite-4                                      1.133Ki ±  0%
List/dqlite-tiny/all-4                              213.3 ±  2%
List/dqlite-tiny/pagination-4                       197.7 ±  1%
List/dqlite-fits-in-page/all-4                    1.321Ki ±  9%
List/dqlite-fits-in-page/pagination-4             1.396Ki ±  5%
List/dqlite-overflows-page/all-4                  7.006Ki ±  4%
List/dqlite-overflows-page/pagination-4           7.094Ki ±  6%
Update/dqlite/1-workers-4                           262.8 ±  0%
Update/dqlite/4-workers-4                           263.2 ±  0%
Update/dqlite/16-workers-4                          264.0 ±  1%
Update/dqlite/64-workers-4                          267.7 ±  2%
Update/dqlite/128-workers-4                         279.3 ±  5%
geomean                                             568.2

                                        │          After           │
                                        │ network-bytes-written/op │
Compaction/dqlite-4                                    4.597 ± 13%
Create/dqlite/1-workers-4                              115.8 ±  0%
Create/dqlite/4-workers-4                              116.6 ±  0%
Create/dqlite/16-workers-4                             117.0 ±  1%
Create/dqlite/64-workers-4                             116.8 ±  1%
Create/dqlite/128-workers-4                            117.5 ±  1%
Delete/dqlite/1-workers-4                              82.26 ±  1%
Delete/dqlite/4-workers-4                              83.00 ±  1%
Delete/dqlite/16-workers-4                             82.75 ±  1%
Delete/dqlite/64-workers-4                             82.88 ±  0%
Delete/dqlite/128-workers-4                            83.21 ± 33%
Get/dqlite-4                                           80.56 ±  0%
List/dqlite-tiny/all-4                               0.04715 ±  3%
List/dqlite-tiny/pagination-4                        0.04871 ±  9%
List/dqlite-fits-in-page/all-4                       0.05951 ± 15%
List/dqlite-fits-in-page/pagination-4                0.08819 ± 13%
List/dqlite-overflows-page/all-4                     0.07914 ±  6%
List/dqlite-overflows-page/pagination-4               0.1228 ± 32%
Update/dqlite/1-workers-4                              136.8 ±  0%
Update/dqlite/4-workers-4                              136.7 ±  0%
Update/dqlite/16-workers-4                             137.1 ±  1%
Update/dqlite/64-workers-4                             139.7 ±  2%
Update/dqlite/128-workers-4                            146.1 ±  6%
geomean                                                13.89

@louiseschmidtgen louiseschmidtgen changed the title Performance-test set-up Performance-test Set-up Nov 6, 2024
@louiseschmidtgen louiseschmidtgen marked this pull request as ready for review November 11, 2024 11:09
@louiseschmidtgen louiseschmidtgen requested a review from a team as a code owner November 11, 2024 11:09
@louiseschmidtgen louiseschmidtgen changed the title Performance-test Set-up Performance Test Set-up (1) Nov 11, 2024
Copy link
Contributor

@HomayoonAlimohammadi HomayoonAlimohammadi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really amazing job @louiseschmidtgen! I tried to go through it as accurate as I could, and LGTM! It should have been such a huge effort to prepare this PR. Wonderful job!

.github/workflows/performance.yaml Outdated Show resolved Hide resolved
Copy link
Contributor

@eaudetcobello eaudetcobello left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work so far! I did an initial pass. I am concerned about code duplication with k8s-snap/tests/integration.

.github/workflows/performance.yaml Outdated Show resolved Hide resolved
.github/workflows/performance.yaml Outdated Show resolved Hide resolved
.github/workflows/performance.yaml Outdated Show resolved Hide resolved
.github/workflows/performance.yaml Outdated Show resolved Hide resolved
.github/workflows/performance.yaml Show resolved Hide resolved
.github/workflows/performance.yaml Outdated Show resolved Hide resolved
.github/workflows/performance.yaml Outdated Show resolved Hide resolved
test/performance/Readme.md Show resolved Hide resolved
test/performance/Readme.md Show resolved Hide resolved
test/performance/lxd-profile.yaml Show resolved Hide resolved
@louiseschmidtgen louiseschmidtgen merged commit 1f9ffa3 into master Nov 12, 2024
9 checks passed
@louiseschmidtgen louiseschmidtgen deleted the KU-1866/performance-test branch November 12, 2024 18:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants