Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Performance Test Parse Metrics (3) #200

Merged
merged 7 commits into from
Nov 14, 2024
Merged

Conversation

louiseschmidtgen
Copy link
Contributor

@louiseschmidtgen louiseschmidtgen commented Nov 12, 2024

Performance Test Parse Metrics

This PR takes care of parsing the metrics from the load test (single & three-node) and uploading the results to an artifact.
(GH does not allow uploading images to GH comments- this was the original plan).

We generate the following files & directories:

Folder Files
head head-1-three-node.log, head-2-three-node.log, head-3-three-node.log , head-1-single-node.log
base-code base-code-1-three-node.log, base-code-2-three-node.log, base-code-3-three-node.log , base-code-1-single-node.log
v1-1-11 ...
v1-2-0 ...

The test creates another directory for the single-node comparison:

Folder Files
single-node head-1-single-node.log, base-code-1-single-node.log, v1-2-0 -1-single-node.log, v1-1-11-1-single-node.log

The script creates graphs for cpu, mem, io reads and writes for all files matching a pattern in a directory.

Here is an example for the head's 3 node cpu performance:
(Kube-burner sleeps for 60 seconds before starting the test)

image

Here is an example for the single-node io writes:
image

These can be retrieved from the CI run's artifact.

Depends on: #199

Copy link

github-actions bot commented Nov 12, 2024

Benchmark Result

goos: linux
goarch: amd64
pkg: github.com/canonical/k8s-dqlite/test
cpu: AMD EPYC 7763 64-Core Processor                
                                        │      Before      │                 After                 │
                                        │      sec/op      │     sec/op       vs base              │
Compaction/sqlite-4                       19.80µ ±      9%    19.87µ ±    8%       ~ (p=0.902 n=7)
Compaction/dqlite-4                       40.40µ ±      4%    42.02µ ±   12%       ~ (p=0.383 n=7)
Create/sqlite/1-workers-4                 504.4µ ±     10%    487.4µ ±    4%       ~ (p=0.805 n=7)
Create/sqlite/4-workers-4                 467.7µ ±      6%    465.9µ ±    6%       ~ (p=0.456 n=7)
Create/sqlite/16-workers-4                424.8µ ±      6%    433.9µ ±    6%       ~ (p=0.097 n=7)
Create/sqlite/64-workers-4                469.1µ ±    388%    486.6µ ±  308%       ~ (p=0.318 n=7)
Create/sqlite/128-workers-4               532.4µ ±     18%    510.2µ ±  266%       ~ (p=1.000 n=7)
Create/dqlite/1-workers-4                 1.072m ±      5%    1.077m ±    6%       ~ (p=0.383 n=7)
Create/dqlite/4-workers-4                 936.6µ ±      7%    928.0µ ±    9%       ~ (p=1.000 n=7)
Create/dqlite/16-workers-4                934.6µ ±      5%    945.6µ ±    5%       ~ (p=0.456 n=7)
Create/dqlite/64-workers-4                952.0µ ±      3%    919.6µ ±    3%  -3.40% (p=0.001 n=7)
Create/dqlite/128-workers-4               931.8µ ±      6%    924.5µ ±    7%       ~ (p=0.620 n=7)
Delete/sqlite/1-workers-4                 478.7µ ±      5%    460.3µ ±    6%       ~ (p=0.053 n=7)
Delete/sqlite/4-workers-4                 461.1µ ±      7%    461.0µ ±    2%       ~ (p=0.318 n=7)
Delete/sqlite/16-workers-4                424.1µ ±     14%    429.8µ ±   11%       ~ (p=1.000 n=7)
Delete/sqlite/64-workers-4                488.3µ ±     86%    465.0µ ± 2164%       ~ (p=1.000 n=7)
Delete/sqlite/128-workers-4               555.1µ ±     59%    540.2µ ±   42%       ~ (p=0.535 n=7)
Delete/dqlite/1-workers-4                 1.067m ±      6%    1.077m ±    5%       ~ (p=0.456 n=7)
Delete/dqlite/4-workers-4                 916.8µ ±      5%    947.1µ ±    4%  +3.31% (p=0.026 n=7)
Delete/dqlite/16-workers-4                963.5µ ±     10%    956.4µ ±    3%       ~ (p=0.318 n=7)
Delete/dqlite/64-workers-4                937.6µ ±      6%    905.5µ ±    5%       ~ (p=0.318 n=7)
Delete/dqlite/128-workers-4               911.9µ ±      6%    936.1µ ±    9%       ~ (p=0.456 n=7)
Get/sqlite-4                              238.5µ ±      2%    233.8µ ±    1%  -1.95% (p=0.001 n=7)
Get/dqlite-4                              357.8µ ±      1%    355.7µ ±    1%       ~ (p=0.318 n=7)
List/sqlite-tiny/all-4                    4.022µ ±      5%    3.895µ ±    5%       ~ (p=0.165 n=7)
List/sqlite-tiny/pagination-4             8.294µ ±      3%    8.416µ ±    2%  +1.47% (p=0.038 n=7)
List/sqlite-fits-in-page/all-4            7.840µ ±      4%    7.734µ ±    6%       ~ (p=0.456 n=7)
List/sqlite-fits-in-page/pagination-4     22.35µ ±      7%    22.59µ ±    5%       ~ (p=0.053 n=7)
List/sqlite-overflows-page/all-4          17.30µ ±      4%    17.37µ ±    3%       ~ (p=0.259 n=7)
List/sqlite-overflows-page/pagination-4   42.70µ ±      7%    43.21µ ±    8%       ~ (p=0.805 n=7)
List/dqlite-tiny/all-4                    2.959µ ±     11%    2.948µ ±    3%       ~ (p=0.710 n=7)
List/dqlite-tiny/pagination-4             6.045µ ±      2%    6.021µ ±    2%       ~ (p=0.735 n=7)
List/dqlite-fits-in-page/all-4            6.861µ ±      2%    6.743µ ±    2%       ~ (p=0.073 n=7)
List/dqlite-fits-in-page/pagination-4     17.46µ ±      2%    17.29µ ±    1%  -0.93% (p=0.011 n=7)
List/dqlite-overflows-page/all-4          21.22µ ±      5%    21.07µ ±    3%       ~ (p=0.259 n=7)
List/dqlite-overflows-page/pagination-4   45.25µ ±      2%    44.43µ ±    2%  -1.81% (p=0.011 n=7)
Update/sqlite/1-workers-4                 483.4µ ±      2%    481.2µ ±    4%       ~ (p=1.000 n=7)
Update/sqlite/4-workers-4                 466.9µ ±      3%    477.2µ ±    4%       ~ (p=0.053 n=7)
Update/sqlite/16-workers-4                440.1µ ±     16%    455.2µ ±   13%       ~ (p=0.383 n=7)
Update/sqlite/64-workers-4                551.6µ ±    604%    458.8µ ±   35%       ~ (p=0.318 n=7)
Update/sqlite/128-workers-4               927.6µ ± 135813%    599.1µ ±  139%       ~ (p=0.073 n=7)
Update/dqlite/1-workers-4                 1.035m ±      4%    1.078m ±    5%  +4.08% (p=0.011 n=7)
Update/dqlite/4-workers-4                 910.6µ ±      9%    952.9µ ±    8%       ~ (p=0.383 n=7)
Update/dqlite/16-workers-4                946.1µ ±      2%    969.2µ ±    5%       ~ (p=0.053 n=7)
Update/dqlite/64-workers-4                944.2µ ±      7%    971.5µ ±    9%       ~ (p=0.165 n=7)
Update/dqlite/128-workers-4               935.2µ ±      9%   1015.1µ ±   10%  +8.54% (p=0.026 n=7)
geomean                                   201.7µ              199.4µ          -1.15%

                                        │        Before        │                    After                     │
                                        │ page-cache-misses/op │ page-cache-misses/op  vs base                │
Compaction/sqlite-4                               0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Compaction/dqlite-4                               0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Create/sqlite/1-workers-4                         0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Create/sqlite/4-workers-4                         0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Create/sqlite/16-workers-4                        0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Create/sqlite/64-workers-4                        0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Create/sqlite/128-workers-4                       0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Create/dqlite/1-workers-4                         0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Create/dqlite/4-workers-4                         0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Create/dqlite/16-workers-4                        0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Create/dqlite/64-workers-4                        0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Create/dqlite/128-workers-4                       0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Delete/sqlite/1-workers-4                         0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Delete/sqlite/4-workers-4                         0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Delete/sqlite/16-workers-4                        0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Delete/sqlite/64-workers-4                        0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Delete/sqlite/128-workers-4                       0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Delete/dqlite/1-workers-4                         0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Delete/dqlite/4-workers-4                         0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Delete/dqlite/16-workers-4                        0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Delete/dqlite/64-workers-4                        0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Delete/dqlite/128-workers-4                       0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Get/sqlite-4                                      0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Get/dqlite-4                                      0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-tiny/all-4                            0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-tiny/pagination-4                     0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-fits-in-page/all-4                    0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-fits-in-page/pagination-4             0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-overflows-page/all-4                  0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-overflows-page/pagination-4           0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-tiny/all-4                            0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-tiny/pagination-4                     0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-fits-in-page/all-4                    0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-fits-in-page/pagination-4             0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-overflows-page/all-4                  0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-overflows-page/pagination-4           0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Update/sqlite/1-workers-4                         0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Update/sqlite/4-workers-4                         0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Update/sqlite/16-workers-4                        0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Update/sqlite/64-workers-4                        0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Update/sqlite/128-workers-4                       0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Update/dqlite/1-workers-4                         0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Update/dqlite/4-workers-4                         0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Update/dqlite/16-workers-4                        0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Update/dqlite/64-workers-4                        0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Update/dqlite/128-workers-4                       0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
geomean                                                      ²                         +0.00%               ²
¹ all samples are equal
² summaries must be >0 to compute geomean

                                        │        Before        │                    After                     │
                                        │ page-cache-spills/op │ page-cache-spills/op  vs base                │
Compaction/sqlite-4                               0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Compaction/dqlite-4                               0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Create/sqlite/1-workers-4                         0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Create/sqlite/4-workers-4                         0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Create/sqlite/16-workers-4                        0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Create/sqlite/64-workers-4                        0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Create/sqlite/128-workers-4                       0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Create/dqlite/1-workers-4                         0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Create/dqlite/4-workers-4                         0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Create/dqlite/16-workers-4                        0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Create/dqlite/64-workers-4                        0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Create/dqlite/128-workers-4                       0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Delete/sqlite/1-workers-4                         0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Delete/sqlite/4-workers-4                         0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Delete/sqlite/16-workers-4                        0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Delete/sqlite/64-workers-4                        0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Delete/sqlite/128-workers-4                       0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Delete/dqlite/1-workers-4                         0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Delete/dqlite/4-workers-4                         0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Delete/dqlite/16-workers-4                        0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Delete/dqlite/64-workers-4                        0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Delete/dqlite/128-workers-4                       0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Get/sqlite-4                                      0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Get/dqlite-4                                      0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-tiny/all-4                            0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-tiny/pagination-4                     0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-fits-in-page/all-4                    0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-fits-in-page/pagination-4             0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-overflows-page/all-4                  0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-overflows-page/pagination-4           0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-tiny/all-4                            0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-tiny/pagination-4                     0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-fits-in-page/all-4                    0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-fits-in-page/pagination-4             0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-overflows-page/all-4                  0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-overflows-page/pagination-4           0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Update/sqlite/1-workers-4                         0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Update/sqlite/4-workers-4                         0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Update/sqlite/16-workers-4                        0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Update/sqlite/64-workers-4                        0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Update/sqlite/128-workers-4                       0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Update/dqlite/1-workers-4                         0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Update/dqlite/4-workers-4                         0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Update/dqlite/16-workers-4                        0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Update/dqlite/64-workers-4                        0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
Update/dqlite/128-workers-4                       0.000 ± 0%               0.000 ± 0%       ~ (p=1.000 n=7) ¹
geomean                                                      ²                         +0.00%               ²
¹ all samples are equal
² summaries must be >0 to compute geomean

                                        │     Before     │                After                │
                                        │ page-reads/op  │ page-reads/op  vs base              │
Compaction/sqlite-4                        12.22 ±    1%     12.23 ±  1%       ~ (p=0.880 n=7)
Compaction/dqlite-4                        12.13 ±    1%     12.22 ±  1%  +0.74% (p=0.009 n=7)
Create/sqlite/1-workers-4                  13.89 ±    0%     13.89 ±  0%       ~ (p=0.938 n=7)
Create/sqlite/4-workers-4                  12.61 ±    2%     12.75 ±  3%       ~ (p=0.209 n=7)
Create/sqlite/16-workers-4                 10.38 ±    4%     10.75 ±  3%       ~ (p=0.091 n=7)
Create/sqlite/64-workers-4                 8.762 ±    5%     8.926 ±  5%       ~ (p=0.209 n=7)
Create/sqlite/128-workers-4                8.243 ±    4%     8.373 ± 12%       ~ (p=0.710 n=7)
Create/dqlite/1-workers-4                  13.45 ±    1%     13.26 ±  2%  -1.41% (p=0.010 n=7)
Create/dqlite/4-workers-4                  11.00 ±    3%     10.87 ±  1%  -1.18% (p=0.043 n=7)
Create/dqlite/16-workers-4                 11.06 ±    1%     10.86 ±  1%  -1.81% (p=0.002 n=7)
Create/dqlite/64-workers-4                 11.00 ±    1%     10.98 ±  1%       ~ (p=0.636 n=7)
Create/dqlite/128-workers-4                10.94 ±    1%     10.96 ±  1%       ~ (p=0.436 n=7)
Delete/sqlite/1-workers-4                  16.23 ±    0%     16.23 ±  0%       ~ (p=0.682 n=7)
Delete/sqlite/4-workers-4                  14.41 ±    3%     14.69 ±  3%       ~ (p=0.073 n=7)
Delete/sqlite/16-workers-4                 11.87 ±    2%     12.09 ± 15%       ~ (p=0.165 n=7)
Delete/sqlite/64-workers-4                10.100 ±    4%     9.997 ± 28%       ~ (p=0.710 n=7)
Delete/sqlite/128-workers-4                9.231 ±    4%     9.311 ±  5%       ~ (p=0.710 n=7)
Delete/dqlite/1-workers-4                  15.80 ±    1%     15.69 ±  2%       ~ (p=0.160 n=7)
Delete/dqlite/4-workers-4                  13.49 ±    1%     13.30 ±  1%  -1.41% (p=0.003 n=7)
Delete/dqlite/16-workers-4                 13.48 ±    1%     13.39 ±  0%  -0.67% (p=0.010 n=7)
Delete/dqlite/64-workers-4                 13.37 ±    4%     13.25 ±  2%       ~ (p=0.300 n=7)
Delete/dqlite/128-workers-4                13.42 ±    2%     13.29 ±  4%       ~ (p=0.154 n=7)
Get/sqlite-4                               14.96 ±    0%     14.95 ±  0%       ~ (p=0.478 n=7)
Get/dqlite-4                               11.96 ±    0%     11.96 ±  0%       ~ (p=1.000 n=7)
List/sqlite-tiny/all-4                    191.7m ±    1%    191.0m ±  1%       ~ (p=0.299 n=7)
List/sqlite-tiny/pagination-4              1.131 ±    3%     1.131 ±  3%       ~ (p=0.731 n=7)
List/sqlite-fits-in-page/all-4            164.7m ±    1%    163.6m ±  2%       ~ (p=0.120 n=7)
List/sqlite-fits-in-page/pagination-4     957.4m ±    3%    962.4m ±  3%       ~ (p=0.684 n=7)
List/sqlite-overflows-page/all-4          177.1m ±    0%    177.1m ±  0%       ~ (p=1.000 n=7)
List/sqlite-overflows-page/pagination-4    1.590 ±   33%     1.597 ± 33%       ~ (p=0.902 n=7)
List/dqlite-tiny/all-4                    191.6m ±    1%    191.5m ±  0%       ~ (p=1.000 n=7)
List/dqlite-tiny/pagination-4              1.126 ±    3%     1.126 ±  3%       ~ (p=0.462 n=7)
List/dqlite-fits-in-page/all-4            164.1m ±    0%    164.0m ±  0%       ~ (p=0.443 n=7)
List/dqlite-fits-in-page/pagination-4     955.6m ±    0%    955.1m ±  0%       ~ (p=0.196 n=7)
List/dqlite-overflows-page/all-4          184.0m ±    0%    184.0m ±  0%       ~ (p=0.984 n=7)
List/dqlite-overflows-page/pagination-4    1.071 ±    0%     1.071 ±  3%       ~ (p=0.122 n=7)
Update/sqlite/1-workers-4                  15.84 ±    0%     15.83 ±  0%       ~ (p=0.462 n=7)
Update/sqlite/4-workers-4                  14.60 ±    3%     14.79 ±  3%  +1.30% (p=0.043 n=7)
Update/sqlite/16-workers-4                 12.74 ±    6%     12.88 ±  7%       ~ (p=0.383 n=7)
Update/sqlite/64-workers-4                 11.31 ±    8%     11.03 ±  3%       ~ (p=0.600 n=7)
Update/sqlite/128-workers-4                11.08 ± 4936%     10.65 ±  7%  -3.88% (p=0.038 n=7)
Update/dqlite/1-workers-4                  15.61 ±    1%     15.41 ±  1%  -1.28% (p=0.006 n=7)
Update/dqlite/4-workers-4                  12.80 ±    2%     12.65 ±  1%  -1.17% (p=0.002 n=7)
Update/dqlite/16-workers-4                 12.90 ±    2%     12.75 ±  2%       ~ (p=0.078 n=7)
Update/dqlite/64-workers-4                 13.09 ±    2%     12.89 ±  2%       ~ (p=0.122 n=7)
Update/dqlite/128-workers-4                13.36 ±    5%     12.99 ±  5%       ~ (p=0.383 n=7)
geomean                                    5.214             5.204        -0.20%

                                        │      Before       │                 After                  │
                                        │  page-writes/op   │ page-writes/op  vs base                │
Compaction/sqlite-4                       190.6m ±     0%        190.8m ± 0%       ~ (p=0.196 n=7)
Compaction/dqlite-4                       187.5m ±     1%        187.9m ± 1%       ~ (p=0.927 n=7)
Create/sqlite/1-workers-4                  4.358 ±     0%         4.359 ± 0%       ~ (p=0.888 n=7)
Create/sqlite/4-workers-4                  4.326 ±     1%         4.323 ± 1%       ~ (p=0.837 n=7)
Create/sqlite/16-workers-4                 4.292 ±     2%         4.282 ± 1%       ~ (p=0.475 n=7)
Create/sqlite/64-workers-4                 4.253 ±     5%         4.234 ± 4%       ~ (p=0.456 n=7)
Create/sqlite/128-workers-4                4.227 ±     1%         4.225 ± 4%       ~ (p=0.620 n=7)
Create/dqlite/1-workers-4                  4.243 ±     1%         4.242 ± 1%       ~ (p=0.777 n=7)
Create/dqlite/4-workers-4                  4.289 ±     1%         4.285 ± 1%       ~ (p=0.646 n=7)
Create/dqlite/16-workers-4                 4.299 ±     0%         4.299 ± 1%       ~ (p=0.300 n=7)
Create/dqlite/64-workers-4                 4.296 ±     0%         4.311 ± 0%       ~ (p=0.135 n=7)
Create/dqlite/128-workers-4                4.295 ±     1%         4.296 ± 1%       ~ (p=0.399 n=7)
Delete/sqlite/1-workers-4                  4.386 ±     0%         4.390 ± 0%       ~ (p=0.277 n=7)
Delete/sqlite/4-workers-4                  4.369 ±     1%         4.383 ± 1%       ~ (p=0.209 n=7)
Delete/sqlite/16-workers-4                 4.350 ±     1%         4.381 ± 1%       ~ (p=0.364 n=7)
Delete/sqlite/64-workers-4                 4.392 ±     2%         4.354 ± 5%       ~ (p=0.258 n=7)
Delete/sqlite/128-workers-4                4.343 ±     1%         4.331 ± 2%       ~ (p=1.000 n=7)
Delete/dqlite/1-workers-4                  4.369 ±     1%         4.363 ± 1%       ~ (p=0.736 n=7)
Delete/dqlite/4-workers-4                  4.377 ±     1%         4.372 ± 0%       ~ (p=0.508 n=7)
Delete/dqlite/16-workers-4                 4.371 ±     0%         4.373 ± 1%       ~ (p=0.636 n=7)
Delete/dqlite/64-workers-4                 4.352 ±     0%         4.355 ± 1%       ~ (p=0.430 n=7)
Delete/dqlite/128-workers-4                4.356 ±     1%         4.359 ± 1%       ~ (p=0.644 n=7)
Get/sqlite-4                               0.000 ±     0%         0.000 ± 0%       ~ (p=1.000 n=7) ¹
Get/dqlite-4                               0.000 ±     0%         0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-tiny/all-4                     0.000 ±     0%         0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-tiny/pagination-4              0.000 ±     0%         0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-fits-in-page/all-4             0.000 ±     0%         0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-fits-in-page/pagination-4      0.000 ±     0%         0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-overflows-page/all-4           0.000 ±     0%         0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/sqlite-overflows-page/pagination-4    0.000 ±     0%         0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-tiny/all-4                     0.000 ±     0%         0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-tiny/pagination-4              0.000 ±     0%         0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-fits-in-page/all-4             0.000 ±     0%         0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-fits-in-page/pagination-4      0.000 ±     0%         0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-overflows-page/all-4           0.000 ±     0%         0.000 ± 0%       ~ (p=1.000 n=7) ¹
List/dqlite-overflows-page/pagination-4    0.000 ±     0%         0.000 ± 0%       ~ (p=1.000 n=7) ¹
Update/sqlite/1-workers-4                  4.248 ±     0%         4.245 ± 0%  -0.07% (p=0.040 n=7)
Update/sqlite/4-workers-4                  4.319 ±     0%         4.314 ± 1%       ~ (p=0.927 n=7)
Update/sqlite/16-workers-4                 4.322 ±     0%         4.336 ± 1%       ~ (p=0.600 n=7)
Update/sqlite/64-workers-4                 4.394 ±     9%         4.375 ± 2%       ~ (p=0.209 n=7)
Update/sqlite/128-workers-4                4.487 ± 11378%         4.423 ± 2%       ~ (p=0.247 n=7)
Update/dqlite/1-workers-4                  4.215 ±     0%         4.205 ± 0%       ~ (p=0.170 n=7)
Update/dqlite/4-workers-4                  4.215 ±     0%         4.209 ± 1%       ~ (p=0.596 n=7)
Update/dqlite/16-workers-4                 4.227 ±     1%         4.239 ± 1%       ~ (p=0.138 n=7)
Update/dqlite/64-workers-4                 4.308 ±     3%         4.360 ± 2%       ~ (p=0.383 n=7)
Update/dqlite/128-workers-4                4.413 ±     5%         4.419 ± 4%       ~ (p=1.000 n=7)
geomean                                                   ²                   -0.02%               ²
¹ all samples are equal
² summaries must be >0 to compute geomean

                                        │     Before      │                 After                 │
                                        │ sec-reading/op  │ sec-reading/op  vs base               │
Compaction/sqlite-4                       26.10µ ±    32%     27.80µ ± 39%        ~ (p=0.555 n=7)
Compaction/dqlite-4                       27.60µ ±    43%     29.90µ ± 73%        ~ (p=0.805 n=7)
Create/sqlite/1-workers-4                 22.50µ ±    30%     20.30µ ± 44%        ~ (p=0.535 n=7)
Create/sqlite/4-workers-4                 19.70µ ±    29%     19.60µ ± 26%        ~ (p=0.927 n=7)
Create/sqlite/16-workers-4                21.50µ ±    17%     20.30µ ± 15%        ~ (p=0.318 n=7)
Create/sqlite/64-workers-4                24.60µ ±    25%     25.50µ ± 39%        ~ (p=0.686 n=7)
Create/sqlite/128-workers-4               23.70µ ±    19%     23.20µ ± 21%        ~ (p=0.367 n=7)
Create/dqlite/1-workers-4                 11.40µ ±    74%     10.00µ ± 51%        ~ (p=0.197 n=7)
Create/dqlite/4-workers-4                 13.20µ ±    50%     12.80µ ± 25%        ~ (p=1.000 n=7)
Create/dqlite/16-workers-4                12.60µ ±    44%     12.60µ ± 60%        ~ (p=0.476 n=7)
Create/dqlite/64-workers-4                10.40µ ±    28%     10.30µ ± 43%        ~ (p=1.000 n=7)
Create/dqlite/128-workers-4               8.000µ ±   105%    12.400µ ± 79%        ~ (p=0.259 n=7)
Delete/sqlite/1-workers-4                 33.40µ ±    13%     32.10µ ± 19%        ~ (p=0.929 n=7)
Delete/sqlite/4-workers-4                 35.80µ ±    18%     35.30µ ± 23%        ~ (p=0.902 n=7)
Delete/sqlite/16-workers-4                33.90µ ±    11%     32.10µ ± 31%        ~ (p=0.364 n=7)
Delete/sqlite/64-workers-4                39.20µ ±    10%     36.00µ ± 26%        ~ (p=0.401 n=7)
Delete/sqlite/128-workers-4               38.40µ ±    15%     34.90µ ± 34%        ~ (p=0.318 n=7)
Delete/dqlite/1-workers-4                 13.70µ ±    35%     14.50µ ± 39%        ~ (p=1.000 n=7)
Delete/dqlite/4-workers-4                 17.70µ ±    41%     17.10µ ± 34%        ~ (p=0.365 n=7)
Delete/dqlite/16-workers-4                16.10µ ±    33%     15.20µ ± 37%        ~ (p=0.209 n=7)
Delete/dqlite/64-workers-4                16.00µ ±    53%     15.70µ ± 45%        ~ (p=0.929 n=7)
Delete/dqlite/128-workers-4               17.90µ ±    41%     15.30µ ± 56%        ~ (p=0.535 n=7)
Get/sqlite-4                              65.80µ ±     4%     64.40µ ±  8%        ~ (p=0.297 n=7)
Get/dqlite-4                              33.80µ ±     9%     33.20µ ± 11%        ~ (p=0.645 n=7)
List/sqlite-tiny/all-4                    6.900µ ±    14%     6.200µ ± 21%        ~ (p=0.300 n=7)
List/sqlite-tiny/pagination-4             14.60µ ±     3%     14.80µ ±  5%        ~ (p=0.091 n=7)
List/sqlite-fits-in-page/all-4            14.40µ ±    22%     12.80µ ± 25%        ~ (p=0.367 n=7)
List/sqlite-fits-in-page/pagination-4     38.40µ ±    24%     38.60µ ± 17%        ~ (p=0.365 n=7)
List/sqlite-overflows-page/all-4          13.40µ ±     3%     13.50µ ±  3%        ~ (p=0.503 n=7)
List/sqlite-overflows-page/pagination-4   47.10µ ±    27%     47.00µ ± 28%        ~ (p=0.837 n=7)
List/dqlite-tiny/all-4                    4.100µ ±    24%     4.100µ ± 20%        ~ (p=0.537 n=7)
List/dqlite-tiny/pagination-4             9.700µ ±    15%     8.900µ ± 13%        ~ (p=0.399 n=7)
List/dqlite-fits-in-page/all-4            11.00µ ±    15%     10.80µ ± 17%        ~ (p=0.682 n=7)
List/dqlite-fits-in-page/pagination-4     28.50µ ±    10%     25.30µ ±  8%  -11.23% (p=0.002 n=7)
List/dqlite-overflows-page/all-4          34.90µ ±     3%     34.70µ ±  3%        ~ (p=0.513 n=7)
List/dqlite-overflows-page/pagination-4   67.50µ ±    14%     67.00µ ±  6%        ~ (p=0.333 n=7)
Update/sqlite/1-workers-4                 19.50µ ±    25%     19.20µ ± 34%        ~ (p=0.831 n=7)
Update/sqlite/4-workers-4                 23.30µ ±    15%     21.80µ ± 17%        ~ (p=0.383 n=7)
Update/sqlite/16-workers-4                22.70µ ±    11%     22.00µ ± 20%        ~ (p=0.477 n=7)
Update/sqlite/64-workers-4                21.40µ ±    26%     24.00µ ± 27%        ~ (p=0.557 n=7)
Update/sqlite/128-workers-4               27.20µ ± 32988%     28.20µ ± 19%        ~ (p=0.902 n=7)
Update/dqlite/1-workers-4                 10.90µ ±    56%     10.40µ ± 31%        ~ (p=0.383 n=7)
Update/dqlite/4-workers-4                 10.70µ ±    44%     11.50µ ± 40%        ~ (p=0.902 n=7)
Update/dqlite/16-workers-4                11.50µ ±    54%     11.70µ ± 21%        ~ (p=0.902 n=7)
Update/dqlite/64-workers-4                14.10µ ±    44%     12.90µ ± 50%        ~ (p=0.646 n=7)
Update/dqlite/128-workers-4               11.00µ ±    61%     11.30µ ± 61%        ~ (p=0.456 n=7)
geomean                                   19.26µ              18.97µ         -1.51%

                                        │      Before       │                  After                  │
                                        │  sec-writing/op   │ sec-writing/op  vs base                 │
Compaction/sqlite-4                       9.500µ ±    23%      9.200µ ±   7%        ~ (p=0.304 n=7)
Compaction/dqlite-4                       6.600µ ±     9%      6.600µ ±   9%        ~ (p=0.922 n=7)
Create/sqlite/1-workers-4                 279.5µ ±    24%      267.4µ ±   4%        ~ (p=0.710 n=7)
Create/sqlite/4-workers-4                 1.434m ±     9%      1.407m ±  13%        ~ (p=0.477 n=7)
Create/sqlite/16-workers-4                5.382m ±     7%      5.240m ±   7%        ~ (p=0.259 n=7)
Create/sqlite/64-workers-4                17.73m ±   102%      18.58m ±  81%        ~ (p=0.097 n=7)
Create/sqlite/128-workers-4               33.37m ±    10%      33.83m ±  60%        ~ (p=0.318 n=7)
Create/dqlite/1-workers-4                 41.20µ ±    23%      45.00µ ±  23%        ~ (p=0.535 n=7)
Create/dqlite/4-workers-4                 50.30µ ±    39%      49.70µ ±  13%        ~ (p=1.000 n=7)
Create/dqlite/16-workers-4                44.30µ ±    25%      49.00µ ±  21%        ~ (p=1.000 n=7)
Create/dqlite/64-workers-4                50.00µ ±    23%      41.30µ ±  20%        ~ (p=0.165 n=7)
Create/dqlite/128-workers-4               48.20µ ±    30%      42.40µ ±  26%        ~ (p=0.073 n=7)
Delete/sqlite/1-workers-4                 242.8µ ±     5%      237.2µ ±   9%        ~ (p=0.777 n=7)
Delete/sqlite/4-workers-4                 1.429m ±     7%      1.378m ±  13%        ~ (p=0.195 n=7)
Delete/sqlite/16-workers-4                5.513m ±     8%      5.315m ±  18%        ~ (p=0.073 n=7)
Delete/sqlite/64-workers-4                18.28m ±     9%      18.70m ± 354%        ~ (p=0.456 n=7)
Delete/sqlite/128-workers-4               34.82m ±     8%      35.08m ±  11%        ~ (p=0.456 n=7)
Delete/dqlite/1-workers-4                 50.60µ ±    33%      40.60µ ±  28%        ~ (p=0.209 n=7)
Delete/dqlite/4-workers-4                 53.60µ ±    26%      49.90µ ±  20%        ~ (p=1.000 n=7)
Delete/dqlite/16-workers-4                52.30µ ±    11%      45.90µ ±  26%  -12.24% (p=0.029 n=7)
Delete/dqlite/64-workers-4                42.10µ ±    32%      45.90µ ±  28%        ~ (p=0.318 n=7)
Delete/dqlite/128-workers-4               46.20µ ±    20%      49.30µ ±  32%        ~ (p=1.000 n=7)
Get/sqlite-4                               0.000 ±     0%       0.000 ±   0%        ~ (p=1.000 n=7) ¹
Get/dqlite-4                               0.000 ±     0%       0.000 ±   0%        ~ (p=1.000 n=7) ¹
List/sqlite-tiny/all-4                     0.000 ±     0%       0.000 ±   0%        ~ (p=1.000 n=7) ¹
List/sqlite-tiny/pagination-4              0.000 ±     0%       0.000 ±   0%        ~ (p=1.000 n=7) ¹
List/sqlite-fits-in-page/all-4             0.000 ±     0%       0.000 ±   0%        ~ (p=1.000 n=7) ¹
List/sqlite-fits-in-page/pagination-4      0.000 ±     0%       0.000 ±   0%        ~ (p=1.000 n=7) ¹
List/sqlite-overflows-page/all-4           0.000 ±     0%       0.000 ±   0%        ~ (p=1.000 n=7) ¹
List/sqlite-overflows-page/pagination-4    0.000 ±     0%       0.000 ±   0%        ~ (p=1.000 n=7) ¹
List/dqlite-tiny/all-4                     0.000 ±     0%       0.000 ±   0%        ~ (p=1.000 n=7) ¹
List/dqlite-tiny/pagination-4              0.000 ±     0%       0.000 ±   0%        ~ (p=1.000 n=7) ¹
List/dqlite-fits-in-page/all-4             0.000 ±     0%       0.000 ±   0%        ~ (p=1.000 n=7) ¹
List/dqlite-fits-in-page/pagination-4      0.000 ±     0%       0.000 ±   0%        ~ (p=1.000 n=7) ¹
List/dqlite-overflows-page/all-4           0.000 ±     0%       0.000 ±   0%        ~ (p=1.000 n=7) ¹
List/dqlite-overflows-page/pagination-4    0.000 ±     0%       0.000 ±   0%        ~ (p=1.000 n=7) ¹
Update/sqlite/1-workers-4                 247.9µ ±     3%      258.5µ ±   5%        ~ (p=0.097 n=7)
Update/sqlite/4-workers-4                 1.429m ±     7%      1.398m ±   8%        ~ (p=0.535 n=7)
Update/sqlite/16-workers-4                5.281m ±     5%      5.347m ±   8%        ~ (p=0.383 n=7)
Update/sqlite/64-workers-4                18.24m ±   162%      18.41m ±   5%        ~ (p=1.000 n=7)
Update/sqlite/128-workers-4               39.69m ± 33158%      34.69m ±  31%        ~ (p=0.165 n=7)
Update/dqlite/1-workers-4                 39.20µ ±    10%      36.20µ ±  19%   -7.65% (p=0.038 n=7)
Update/dqlite/4-workers-4                 47.60µ ±    18%      39.50µ ±  13%  -17.02% (p=0.011 n=7)
Update/dqlite/16-workers-4                48.50µ ±    23%      40.70µ ±  33%        ~ (p=0.710 n=7)
Update/dqlite/64-workers-4                43.60µ ±    13%      47.20µ ±  25%        ~ (p=0.318 n=7)
Update/dqlite/128-workers-4               43.80µ ±    22%      44.20µ ±  27%        ~ (p=1.000 n=7)
geomean                                                   ²                    -2.12%               ²
¹ all samples are equal
² summaries must be >0 to compute geomean

                                        │      Before      │                After                │
                                        │       B/op       │     B/op       vs base              │
Compaction/sqlite-4                       2.212Ki ±    38%   2.353Ki ± 49%       ~ (p=0.533 n=7)
Compaction/dqlite-4                       4.893Ki ±    60%   5.180Ki ± 90%       ~ (p=0.620 n=7)
Create/sqlite/1-workers-4                 20.49Ki ±     0%   20.50Ki ±  0%       ~ (p=0.730 n=7)
Create/sqlite/4-workers-4                 20.49Ki ±     0%   20.49Ki ±  0%       ~ (p=0.433 n=7)
Create/sqlite/16-workers-4                20.47Ki ±     0%   20.47Ki ±  0%       ~ (p=0.876 n=7)
Create/sqlite/64-workers-4                20.47Ki ±     4%   20.52Ki ±  2%       ~ (p=0.154 n=7)
Create/sqlite/128-workers-4               20.58Ki ±     0%   20.57Ki ±  2%       ~ (p=0.686 n=7)
Create/dqlite/1-workers-4                 19.75Ki ±     0%   19.75Ki ±  0%       ~ (p=0.402 n=7)
Create/dqlite/4-workers-4                 19.72Ki ±     0%   19.72Ki ±  0%       ~ (p=0.925 n=7)
Create/dqlite/16-workers-4                19.72Ki ±     0%   19.73Ki ±  0%       ~ (p=0.824 n=7)
Create/dqlite/64-workers-4                19.74Ki ±     0%   19.72Ki ±  1%  -0.08% (p=0.002 n=7)
Create/dqlite/128-workers-4               19.75Ki ±    11%   19.74Ki ± 11%       ~ (p=0.269 n=7)
Delete/sqlite/1-workers-4                 22.93Ki ±     0%   22.92Ki ±  0%       ~ (p=0.154 n=7)
Delete/sqlite/4-workers-4                 22.91Ki ±     0%   22.92Ki ±  0%       ~ (p=0.511 n=7)
Delete/sqlite/16-workers-4                22.89Ki ±     0%   22.89Ki ±  0%       ~ (p=0.869 n=7)
Delete/sqlite/64-workers-4                22.91Ki ±     0%   22.92Ki ±  9%       ~ (p=0.902 n=7)
Delete/sqlite/128-workers-4               23.04Ki ±     0%   23.02Ki ±  0%       ~ (p=0.737 n=7)
Delete/dqlite/1-workers-4                 22.01Ki ±     0%   22.02Ki ±  0%       ~ (p=0.333 n=7)
Delete/dqlite/4-workers-4                 21.98Ki ±     0%   21.99Ki ±  0%       ~ (p=0.784 n=7)
Delete/dqlite/16-workers-4                21.99Ki ±     0%   21.99Ki ±  0%       ~ (p=0.710 n=7)
Delete/dqlite/64-workers-4                21.99Ki ±    10%   21.99Ki ± 10%       ~ (p=0.334 n=7)
Delete/dqlite/128-workers-4               22.02Ki ±     0%   22.01Ki ± 10%       ~ (p=0.274 n=7)
Get/sqlite-4                              24.42Ki ±     0%   24.42Ki ±  0%       ~ (p=0.783 n=7)
Get/dqlite-4                              23.25Ki ±     0%   23.25Ki ±  0%       ~ (p=0.714 n=7)
List/sqlite-tiny/all-4                    1.207Ki ±    13%   1.142Ki ± 15%       ~ (p=0.245 n=7)
List/sqlite-tiny/pagination-4             1.556Ki ±     2%   1.580Ki ±  8%       ~ (p=0.535 n=7)
List/sqlite-fits-in-page/all-4            7.800Ki ±    15%   7.337Ki ± 22%       ~ (p=0.318 n=7)
List/sqlite-fits-in-page/pagination-4     10.75Ki ±    27%   10.52Ki ± 14%       ~ (p=0.620 n=7)
List/sqlite-overflows-page/all-4          20.54Ki ±     0%   20.54Ki ±  0%       ~ (p=0.435 n=7)
List/sqlite-overflows-page/pagination-4   28.29Ki ±    40%   29.50Ki ± 42%       ~ (p=0.805 n=7)
List/dqlite-tiny/all-4                    1.062Ki ±    10%   1.061Ki ±  6%       ~ (p=0.781 n=7)
List/dqlite-tiny/pagination-4               973.0 ±     2%     981.0 ±  5%       ~ (p=0.535 n=7)
List/dqlite-fits-in-page/all-4            6.069Ki ±     9%   6.031Ki ±  6%       ~ (p=0.301 n=7)
List/dqlite-fits-in-page/pagination-4     5.558Ki ±     7%   5.426Ki ±  5%       ~ (p=0.097 n=7)
List/dqlite-overflows-page/all-4          30.11Ki ±     6%   30.20Ki ±  8%       ~ (p=0.710 n=7)
List/dqlite-overflows-page/pagination-4   29.32Ki ±     7%   29.27Ki ±  7%       ~ (p=0.383 n=7)
Update/sqlite/1-workers-4                 22.28Ki ±     0%   22.28Ki ±  0%       ~ (p=0.876 n=7)
Update/sqlite/4-workers-4                 22.29Ki ±     0%   22.29Ki ±  0%       ~ (p=1.000 n=7)
Update/sqlite/16-workers-4                22.37Ki ±     0%   22.38Ki ±  0%       ~ (p=0.332 n=7)
Update/sqlite/64-workers-4                22.75Ki ±    18%   22.63Ki ±  1%       ~ (p=0.128 n=7)
Update/sqlite/128-workers-4               23.62Ki ± 12346%   23.15Ki ±  3%       ~ (p=0.165 n=7)
Update/dqlite/1-workers-4                 21.48Ki ±     0%   21.49Ki ±  0%  +0.05% (p=0.035 n=7)
Update/dqlite/4-workers-4                 21.49Ki ±     0%   21.46Ki ±  0%       ~ (p=0.172 n=7)
Update/dqlite/16-workers-4                21.53Ki ±     0%   21.69Ki ±  1%       ~ (p=0.053 n=7)
Update/dqlite/64-workers-4                21.82Ki ±     2%   22.11Ki ±  2%       ~ (p=0.259 n=7)
Update/dqlite/128-workers-4               22.26Ki ±    13%   22.55Ki ±  4%       ~ (p=0.644 n=7)
geomean                                   14.37Ki            14.37Ki        +0.05%

                                        │     Before     │                After                │
                                        │   allocs/op    │  allocs/op   vs base                │
Compaction/sqlite-4                       47.00 ±    38%   50.00 ± 50%       ~ (p=0.642 n=7)
Compaction/dqlite-4                       117.0 ±    61%   124.0 ± 91%       ~ (p=0.596 n=7)
Create/sqlite/1-workers-4                 399.0 ±     0%   399.0 ±  0%       ~ (p=1.000 n=7) ¹
Create/sqlite/4-workers-4                 399.0 ±     0%   399.0 ±  0%       ~ (p=1.000 n=7) ¹
Create/sqlite/16-workers-4                399.0 ±     0%   399.0 ±  0%       ~ (p=1.000 n=7)
Create/sqlite/64-workers-4                398.0 ±     1%   399.0 ±  1%  +0.25% (p=0.049 n=7)
Create/sqlite/128-workers-4               399.0 ±     0%   400.0 ±  1%       ~ (p=0.603 n=7)
Create/dqlite/1-workers-4                 369.0 ±     0%   369.0 ±  0%       ~ (p=1.000 n=7) ¹
Create/dqlite/4-workers-4                 369.0 ±     0%   369.0 ±  0%       ~ (p=1.000 n=7)
Create/dqlite/16-workers-4                369.0 ±     0%   369.0 ±  0%       ~ (p=1.000 n=7)
Create/dqlite/64-workers-4                368.0 ±     0%   368.0 ±  1%       ~ (p=1.000 n=7)
Create/dqlite/128-workers-4               368.0 ±    14%   368.0 ± 14%       ~ (p=1.000 n=7)
Delete/sqlite/1-workers-4                 431.0 ±     0%   431.0 ±  0%       ~ (p=1.000 n=7) ¹
Delete/sqlite/4-workers-4                 431.0 ±     0%   431.0 ±  0%       ~ (p=1.000 n=7)
Delete/sqlite/16-workers-4                430.0 ±     0%   430.0 ±  0%       ~ (p=1.000 n=7)
Delete/sqlite/64-workers-4                430.0 ±     0%   430.0 ±  0%       ~ (p=1.000 n=7)
Delete/sqlite/128-workers-4               431.0 ±     0%   431.0 ±  0%       ~ (p=1.000 n=7)
Delete/dqlite/1-workers-4                 399.0 ±     0%   399.0 ±  0%       ~ (p=1.000 n=7) ¹
Delete/dqlite/4-workers-4                 399.0 ±     0%   399.0 ±  0%       ~ (p=1.000 n=7) ¹
Delete/dqlite/16-workers-4                399.0 ±     0%   399.0 ±  0%       ~ (p=1.000 n=7) ¹
Delete/dqlite/64-workers-4                399.0 ±    12%   399.0 ± 13%       ~ (p=0.339 n=7)
Delete/dqlite/128-workers-4               399.0 ±     0%   399.0 ± 13%       ~ (p=1.000 n=7)
Get/sqlite-4                              445.0 ±     0%   445.0 ±  0%       ~ (p=1.000 n=7) ¹
Get/dqlite-4                              412.0 ±     0%   412.0 ±  0%       ~ (p=1.000 n=7) ¹
List/sqlite-tiny/all-4                    16.00 ±    12%   15.00 ± 20%       ~ (p=0.386 n=7)
List/sqlite-tiny/pagination-4             25.00 ±     4%   25.00 ±  8%       ~ (p=0.147 n=7)
List/sqlite-fits-in-page/all-4            23.00 ±    26%   20.00 ± 30%       ~ (p=0.233 n=7)
List/sqlite-fits-in-page/pagination-4     39.00 ±    36%   38.00 ± 18%       ~ (p=0.781 n=7)
List/sqlite-overflows-page/all-4          9.000 ±     0%   9.000 ±  0%       ~ (p=1.000 n=7) ¹
List/sqlite-overflows-page/pagination-4   18.00 ±    56%   18.00 ± 50%       ~ (p=0.883 n=7)
List/dqlite-tiny/all-4                    15.00 ±    20%   15.00 ±  7%       ~ (p=1.000 n=7)
List/dqlite-tiny/pagination-4             14.00 ±     7%   14.00 ±  7%       ~ (p=1.000 n=7)
List/dqlite-fits-in-page/all-4            20.00 ±     5%   19.00 ±  5%       ~ (p=0.592 n=7)
List/dqlite-fits-in-page/pagination-4     20.00 ±     5%   19.00 ±  0%  -5.00% (p=0.021 n=7)
List/dqlite-overflows-page/all-4          33.00 ±     3%   33.00 ±  3%       ~ (p=1.000 n=7)
List/dqlite-overflows-page/pagination-4   34.00 ±     9%   33.00 ±  3%       ~ (p=0.122 n=7)
Update/sqlite/1-workers-4                 412.0 ±     0%   412.0 ±  0%       ~ (p=1.000 n=7) ¹
Update/sqlite/4-workers-4                 412.0 ±     0%   412.0 ±  0%       ~ (p=1.000 n=7) ¹
Update/sqlite/16-workers-4                413.0 ±     0%   413.0 ±  0%       ~ (p=0.298 n=7)
Update/sqlite/64-workers-4                420.0 ±    16%   418.0 ±  1%       ~ (p=0.279 n=7)
Update/sqlite/128-workers-4               436.0 ± 12170%   427.0 ±  3%       ~ (p=0.138 n=7)
Update/dqlite/1-workers-4                 381.0 ±     0%   381.0 ±  0%       ~ (p=1.000 n=7) ¹
Update/dqlite/4-workers-4                 382.0 ±     0%   381.0 ±  0%       ~ (p=0.592 n=7)
Update/dqlite/16-workers-4                382.0 ±     1%   385.0 ±  1%       ~ (p=0.091 n=7)
Update/dqlite/64-workers-4                387.0 ±     2%   392.0 ±  2%       ~ (p=0.371 n=7)
Update/dqlite/128-workers-4               395.0 ±    16%   397.0 ±  5%       ~ (p=0.737 n=7)
geomean                                   171.6            170.7        -0.52%
¹ all samples are equal

                                        │        Before         │                    After                    │
                                        │ network-bytes-read/op │ network-bytes-read/op  vs base              │
Compaction/dqlite-4                               1.765Ki ± 61%           1.871Ki ± 92%       ~ (p=0.620 n=7)
Create/dqlite/1-workers-4                           241.1 ±  0%             240.9 ±  0%       ~ (p=0.651 n=7)
Create/dqlite/4-workers-4                           241.8 ±  0%             241.7 ±  0%       ~ (p=0.474 n=7)
Create/dqlite/16-workers-4                          241.9 ±  0%             242.0 ±  0%       ~ (p=0.890 n=7)
Create/dqlite/64-workers-4                          241.8 ±  0%             242.4 ±  4%       ~ (p=0.055 n=7)
Create/dqlite/128-workers-4                         241.7 ± 35%             242.4 ± 35%  +0.29% (p=0.033 n=7)
Delete/dqlite/1-workers-4                           718.1 ±  0%             717.8 ±  0%       ~ (p=0.315 n=7)
Delete/dqlite/4-workers-4                           719.3 ±  0%             719.2 ±  0%       ~ (p=0.688 n=7)
Delete/dqlite/16-workers-4                          719.1 ±  0%             719.3 ±  0%       ~ (p=0.645 n=7)
Delete/dqlite/64-workers-4                          718.4 ± 12%             718.5 ± 12%       ~ (p=0.559 n=7)
Delete/dqlite/128-workers-4                         719.7 ±  0%             719.2 ± 12%       ~ (p=0.971 n=7)
Get/dqlite-4                                      1.133Ki ±  0%           1.133Ki ±  0%       ~ (p=1.000 n=7)
List/dqlite-tiny/all-4                              213.5 ± 18%             213.2 ±  3%       ~ (p=1.000 n=7)
List/dqlite-tiny/pagination-4                       197.0 ±  3%             198.0 ±  3%       ~ (p=0.620 n=7)
List/dqlite-fits-in-page/all-4                    1.355Ki ±  2%           1.341Ki ±  4%       ~ (p=0.397 n=7)
List/dqlite-fits-in-page/pagination-4             1.397Ki ±  7%           1.306Ki ±  3%  -6.57% (p=0.005 n=7)
List/dqlite-overflows-page/all-4                  6.987Ki ±  3%           6.971Ki ±  2%       ~ (p=1.000 n=7)
List/dqlite-overflows-page/pagination-4           7.220Ki ±  7%           6.849Ki ±  4%       ~ (p=0.053 n=7)
Update/dqlite/1-workers-4                           262.9 ±  0%             262.8 ±  0%       ~ (p=0.070 n=7)
Update/dqlite/4-workers-4                           263.4 ±  0%             263.2 ±  0%       ~ (p=0.106 n=7)
Update/dqlite/16-workers-4                          263.3 ±  0%             265.3 ±  1%       ~ (p=0.058 n=7)
Update/dqlite/64-workers-4                          266.7 ±  5%             270.4 ±  2%       ~ (p=0.365 n=7)
Update/dqlite/128-workers-4                         272.5 ± 35%             273.9 ±  4%       ~ (p=0.710 n=7)
geomean                                             564.5                   563.5        -0.19%

                                        │          Before          │                     After                      │
                                        │ network-bytes-written/op │ network-bytes-written/op  vs base              │
Compaction/dqlite-4                                    4.529 ± 10%                4.532 ± 14%       ~ (p=0.805 n=7)
Create/dqlite/1-workers-4                              115.3 ±  1%                115.1 ±  0%       ~ (p=0.510 n=7)
Create/dqlite/4-workers-4                              116.7 ±  1%                116.5 ±  2%       ~ (p=0.515 n=7)
Create/dqlite/16-workers-4                             116.9 ±  1%                117.1 ±  1%       ~ (p=0.973 n=7)
Create/dqlite/64-workers-4                             116.8 ±  1%                117.7 ±  2%       ~ (p=0.055 n=7)
Create/dqlite/128-workers-4                            116.5 ± 24%                117.7 ± 23%  +1.03% (p=0.030 n=7)
Delete/dqlite/1-workers-4                              82.64 ±  1%                82.31 ±  1%       ~ (p=0.383 n=7)
Delete/dqlite/4-workers-4                              83.11 ±  0%                83.08 ±  0%       ~ (p=0.559 n=7)
Delete/dqlite/16-workers-4                             82.93 ±  0%                83.09 ±  1%       ~ (p=0.334 n=7)
Delete/dqlite/64-workers-4                             82.75 ± 33%                82.80 ± 35%       ~ (p=0.710 n=7)
Delete/dqlite/128-workers-4                            83.25 ±  1%                83.09 ± 34%       ~ (p=0.902 n=7)
Get/dqlite-4                                           80.57 ±  0%                80.50 ±  0%       ~ (p=0.067 n=7)
List/dqlite-tiny/all-4                               0.04705 ± 32%              0.04705 ±  5%       ~ (p=0.831 n=7)
List/dqlite-tiny/pagination-4                        0.05056 ±  6%              0.04865 ± 11%       ~ (p=0.383 n=7)
List/dqlite-fits-in-page/all-4                       0.06195 ±  2%              0.06089 ±  6%       ~ (p=0.557 n=7)
List/dqlite-fits-in-page/pagination-4                0.09488 ± 19%              0.08282 ± 14%       ~ (p=0.073 n=7)
List/dqlite-overflows-page/all-4                     0.07817 ±  5%              0.07805 ±  3%       ~ (p=0.805 n=7)
List/dqlite-overflows-page/pagination-4               0.1317 ± 14%               0.1205 ± 10%       ~ (p=0.318 n=7)
Update/dqlite/1-workers-4                              136.7 ±  0%                136.6 ±  0%       ~ (p=0.862 n=7)
Update/dqlite/4-workers-4                              136.8 ±  0%                136.7 ±  0%       ~ (p=0.233 n=7)
Update/dqlite/16-workers-4                             136.9 ±  1%                137.7 ±  1%  +0.58% (p=0.006 n=7)
Update/dqlite/64-workers-4                             138.8 ±  3%                140.9 ±  2%       ~ (p=0.318 n=7)
Update/dqlite/128-workers-4                            141.9 ± 24%                143.3 ±  4%       ~ (p=0.691 n=7)
geomean                                                13.98                      13.84        -1.04%
Current status
goos: linux
goarch: amd64
pkg: github.com/canonical/k8s-dqlite/test
cpu: AMD EPYC 7763 64-Core Processor                
                                        │     After      │
                                        │     sec/op     │
Compaction/sqlite-4                       19.87µ ±    8%
Compaction/dqlite-4                       42.02µ ±   12%
Create/sqlite/1-workers-4                 487.4µ ±    4%
Create/sqlite/4-workers-4                 465.9µ ±    6%
Create/sqlite/16-workers-4                433.9µ ±    6%
Create/sqlite/64-workers-4                486.6µ ±  308%
Create/sqlite/128-workers-4               510.2µ ±  266%
Create/dqlite/1-workers-4                 1.077m ±    6%
Create/dqlite/4-workers-4                 928.0µ ±    9%
Create/dqlite/16-workers-4                945.6µ ±    5%
Create/dqlite/64-workers-4                919.6µ ±    3%
Create/dqlite/128-workers-4               924.5µ ±    7%
Delete/sqlite/1-workers-4                 460.3µ ±    6%
Delete/sqlite/4-workers-4                 461.0µ ±    2%
Delete/sqlite/16-workers-4                429.8µ ±   11%
Delete/sqlite/64-workers-4                465.0µ ± 2164%
Delete/sqlite/128-workers-4               540.2µ ±   42%
Delete/dqlite/1-workers-4                 1.077m ±    5%
Delete/dqlite/4-workers-4                 947.1µ ±    4%
Delete/dqlite/16-workers-4                956.4µ ±    3%
Delete/dqlite/64-workers-4                905.5µ ±    5%
Delete/dqlite/128-workers-4               936.1µ ±    9%
Get/sqlite-4                              233.8µ ±    1%
Get/dqlite-4                              355.7µ ±    1%
List/sqlite-tiny/all-4                    3.895µ ±    5%
List/sqlite-tiny/pagination-4             8.416µ ±    2%
List/sqlite-fits-in-page/all-4            7.734µ ±    6%
List/sqlite-fits-in-page/pagination-4     22.59µ ±    5%
List/sqlite-overflows-page/all-4          17.37µ ±    3%
List/sqlite-overflows-page/pagination-4   43.21µ ±    8%
List/dqlite-tiny/all-4                    2.948µ ±    3%
List/dqlite-tiny/pagination-4             6.021µ ±    2%
List/dqlite-fits-in-page/all-4            6.743µ ±    2%
List/dqlite-fits-in-page/pagination-4     17.29µ ±    1%
List/dqlite-overflows-page/all-4          21.07µ ±    3%
List/dqlite-overflows-page/pagination-4   44.43µ ±    2%
Update/sqlite/1-workers-4                 481.2µ ±    4%
Update/sqlite/4-workers-4                 477.2µ ±    4%
Update/sqlite/16-workers-4                455.2µ ±   13%
Update/sqlite/64-workers-4                458.8µ ±   35%
Update/sqlite/128-workers-4               599.1µ ±  139%
Update/dqlite/1-workers-4                 1.078m ±    5%
Update/dqlite/4-workers-4                 952.9µ ±    8%
Update/dqlite/16-workers-4                969.2µ ±    5%
Update/dqlite/64-workers-4                971.5µ ±    9%
Update/dqlite/128-workers-4               1.015m ±   10%
geomean                                   199.4µ

                                        │        After         │
                                        │ page-cache-misses/op │
Compaction/sqlite-4                               0.000 ± 0%
Compaction/dqlite-4                               0.000 ± 0%
Create/sqlite/1-workers-4                         0.000 ± 0%
Create/sqlite/4-workers-4                         0.000 ± 0%
Create/sqlite/16-workers-4                        0.000 ± 0%
Create/sqlite/64-workers-4                        0.000 ± 0%
Create/sqlite/128-workers-4                       0.000 ± 0%
Create/dqlite/1-workers-4                         0.000 ± 0%
Create/dqlite/4-workers-4                         0.000 ± 0%
Create/dqlite/16-workers-4                        0.000 ± 0%
Create/dqlite/64-workers-4                        0.000 ± 0%
Create/dqlite/128-workers-4                       0.000 ± 0%
Delete/sqlite/1-workers-4                         0.000 ± 0%
Delete/sqlite/4-workers-4                         0.000 ± 0%
Delete/sqlite/16-workers-4                        0.000 ± 0%
Delete/sqlite/64-workers-4                        0.000 ± 0%
Delete/sqlite/128-workers-4                       0.000 ± 0%
Delete/dqlite/1-workers-4                         0.000 ± 0%
Delete/dqlite/4-workers-4                         0.000 ± 0%
Delete/dqlite/16-workers-4                        0.000 ± 0%
Delete/dqlite/64-workers-4                        0.000 ± 0%
Delete/dqlite/128-workers-4                       0.000 ± 0%
Get/sqlite-4                                      0.000 ± 0%
Get/dqlite-4                                      0.000 ± 0%
List/sqlite-tiny/all-4                            0.000 ± 0%
List/sqlite-tiny/pagination-4                     0.000 ± 0%
List/sqlite-fits-in-page/all-4                    0.000 ± 0%
List/sqlite-fits-in-page/pagination-4             0.000 ± 0%
List/sqlite-overflows-page/all-4                  0.000 ± 0%
List/sqlite-overflows-page/pagination-4           0.000 ± 0%
List/dqlite-tiny/all-4                            0.000 ± 0%
List/dqlite-tiny/pagination-4                     0.000 ± 0%
List/dqlite-fits-in-page/all-4                    0.000 ± 0%
List/dqlite-fits-in-page/pagination-4             0.000 ± 0%
List/dqlite-overflows-page/all-4                  0.000 ± 0%
List/dqlite-overflows-page/pagination-4           0.000 ± 0%
Update/sqlite/1-workers-4                         0.000 ± 0%
Update/sqlite/4-workers-4                         0.000 ± 0%
Update/sqlite/16-workers-4                        0.000 ± 0%
Update/sqlite/64-workers-4                        0.000 ± 0%
Update/sqlite/128-workers-4                       0.000 ± 0%
Update/dqlite/1-workers-4                         0.000 ± 0%
Update/dqlite/4-workers-4                         0.000 ± 0%
Update/dqlite/16-workers-4                        0.000 ± 0%
Update/dqlite/64-workers-4                        0.000 ± 0%
Update/dqlite/128-workers-4                       0.000 ± 0%
geomean                                                      ¹
¹ summaries must be >0 to compute geomean

                                        │        After         │
                                        │ page-cache-spills/op │
Compaction/sqlite-4                               0.000 ± 0%
Compaction/dqlite-4                               0.000 ± 0%
Create/sqlite/1-workers-4                         0.000 ± 0%
Create/sqlite/4-workers-4                         0.000 ± 0%
Create/sqlite/16-workers-4                        0.000 ± 0%
Create/sqlite/64-workers-4                        0.000 ± 0%
Create/sqlite/128-workers-4                       0.000 ± 0%
Create/dqlite/1-workers-4                         0.000 ± 0%
Create/dqlite/4-workers-4                         0.000 ± 0%
Create/dqlite/16-workers-4                        0.000 ± 0%
Create/dqlite/64-workers-4                        0.000 ± 0%
Create/dqlite/128-workers-4                       0.000 ± 0%
Delete/sqlite/1-workers-4                         0.000 ± 0%
Delete/sqlite/4-workers-4                         0.000 ± 0%
Delete/sqlite/16-workers-4                        0.000 ± 0%
Delete/sqlite/64-workers-4                        0.000 ± 0%
Delete/sqlite/128-workers-4                       0.000 ± 0%
Delete/dqlite/1-workers-4                         0.000 ± 0%
Delete/dqlite/4-workers-4                         0.000 ± 0%
Delete/dqlite/16-workers-4                        0.000 ± 0%
Delete/dqlite/64-workers-4                        0.000 ± 0%
Delete/dqlite/128-workers-4                       0.000 ± 0%
Get/sqlite-4                                      0.000 ± 0%
Get/dqlite-4                                      0.000 ± 0%
List/sqlite-tiny/all-4                            0.000 ± 0%
List/sqlite-tiny/pagination-4                     0.000 ± 0%
List/sqlite-fits-in-page/all-4                    0.000 ± 0%
List/sqlite-fits-in-page/pagination-4             0.000 ± 0%
List/sqlite-overflows-page/all-4                  0.000 ± 0%
List/sqlite-overflows-page/pagination-4           0.000 ± 0%
List/dqlite-tiny/all-4                            0.000 ± 0%
List/dqlite-tiny/pagination-4                     0.000 ± 0%
List/dqlite-fits-in-page/all-4                    0.000 ± 0%
List/dqlite-fits-in-page/pagination-4             0.000 ± 0%
List/dqlite-overflows-page/all-4                  0.000 ± 0%
List/dqlite-overflows-page/pagination-4           0.000 ± 0%
Update/sqlite/1-workers-4                         0.000 ± 0%
Update/sqlite/4-workers-4                         0.000 ± 0%
Update/sqlite/16-workers-4                        0.000 ± 0%
Update/sqlite/64-workers-4                        0.000 ± 0%
Update/sqlite/128-workers-4                       0.000 ± 0%
Update/dqlite/1-workers-4                         0.000 ± 0%
Update/dqlite/4-workers-4                         0.000 ± 0%
Update/dqlite/16-workers-4                        0.000 ± 0%
Update/dqlite/64-workers-4                        0.000 ± 0%
Update/dqlite/128-workers-4                       0.000 ± 0%
geomean                                                      ¹
¹ summaries must be >0 to compute geomean

                                        │     After     │
                                        │ page-reads/op │
Compaction/sqlite-4                         12.23 ±  1%
Compaction/dqlite-4                         12.22 ±  1%
Create/sqlite/1-workers-4                   13.89 ±  0%
Create/sqlite/4-workers-4                   12.75 ±  3%
Create/sqlite/16-workers-4                  10.75 ±  3%
Create/sqlite/64-workers-4                  8.926 ±  5%
Create/sqlite/128-workers-4                 8.373 ± 12%
Create/dqlite/1-workers-4                   13.26 ±  2%
Create/dqlite/4-workers-4                   10.87 ±  1%
Create/dqlite/16-workers-4                  10.86 ±  1%
Create/dqlite/64-workers-4                  10.98 ±  1%
Create/dqlite/128-workers-4                 10.96 ±  1%
Delete/sqlite/1-workers-4                   16.23 ±  0%
Delete/sqlite/4-workers-4                   14.69 ±  3%
Delete/sqlite/16-workers-4                  12.09 ± 15%
Delete/sqlite/64-workers-4                  9.997 ± 28%
Delete/sqlite/128-workers-4                 9.311 ±  5%
Delete/dqlite/1-workers-4                   15.69 ±  2%
Delete/dqlite/4-workers-4                   13.30 ±  1%
Delete/dqlite/16-workers-4                  13.39 ±  0%
Delete/dqlite/64-workers-4                  13.25 ±  2%
Delete/dqlite/128-workers-4                 13.29 ±  4%
Get/sqlite-4                                14.95 ±  0%
Get/dqlite-4                                11.96 ±  0%
List/sqlite-tiny/all-4                     191.0m ±  1%
List/sqlite-tiny/pagination-4               1.131 ±  3%
List/sqlite-fits-in-page/all-4             163.6m ±  2%
List/sqlite-fits-in-page/pagination-4      962.4m ±  3%
List/sqlite-overflows-page/all-4           177.1m ±  0%
List/sqlite-overflows-page/pagination-4     1.597 ± 33%
List/dqlite-tiny/all-4                     191.5m ±  0%
List/dqlite-tiny/pagination-4               1.126 ±  3%
List/dqlite-fits-in-page/all-4             164.0m ±  0%
List/dqlite-fits-in-page/pagination-4      955.1m ±  0%
List/dqlite-overflows-page/all-4           184.0m ±  0%
List/dqlite-overflows-page/pagination-4     1.071 ±  3%
Update/sqlite/1-workers-4                   15.83 ±  0%
Update/sqlite/4-workers-4                   14.79 ±  3%
Update/sqlite/16-workers-4                  12.88 ±  7%
Update/sqlite/64-workers-4                  11.03 ±  3%
Update/sqlite/128-workers-4                 10.65 ±  7%
Update/dqlite/1-workers-4                   15.41 ±  1%
Update/dqlite/4-workers-4                   12.65 ±  1%
Update/dqlite/16-workers-4                  12.75 ±  2%
Update/dqlite/64-workers-4                  12.89 ±  2%
Update/dqlite/128-workers-4                 12.99 ±  5%
geomean                                     5.204

                                        │     After      │
                                        │ page-writes/op │
Compaction/sqlite-4                        190.8m ± 0%
Compaction/dqlite-4                        187.9m ± 1%
Create/sqlite/1-workers-4                   4.359 ± 0%
Create/sqlite/4-workers-4                   4.323 ± 1%
Create/sqlite/16-workers-4                  4.282 ± 1%
Create/sqlite/64-workers-4                  4.234 ± 4%
Create/sqlite/128-workers-4                 4.225 ± 4%
Create/dqlite/1-workers-4                   4.242 ± 1%
Create/dqlite/4-workers-4                   4.285 ± 1%
Create/dqlite/16-workers-4                  4.299 ± 1%
Create/dqlite/64-workers-4                  4.311 ± 0%
Create/dqlite/128-workers-4                 4.296 ± 1%
Delete/sqlite/1-workers-4                   4.390 ± 0%
Delete/sqlite/4-workers-4                   4.383 ± 1%
Delete/sqlite/16-workers-4                  4.381 ± 1%
Delete/sqlite/64-workers-4                  4.354 ± 5%
Delete/sqlite/128-workers-4                 4.331 ± 2%
Delete/dqlite/1-workers-4                   4.363 ± 1%
Delete/dqlite/4-workers-4                   4.372 ± 0%
Delete/dqlite/16-workers-4                  4.373 ± 1%
Delete/dqlite/64-workers-4                  4.355 ± 1%
Delete/dqlite/128-workers-4                 4.359 ± 1%
Get/sqlite-4                                0.000 ± 0%
Get/dqlite-4                                0.000 ± 0%
List/sqlite-tiny/all-4                      0.000 ± 0%
List/sqlite-tiny/pagination-4               0.000 ± 0%
List/sqlite-fits-in-page/all-4              0.000 ± 0%
List/sqlite-fits-in-page/pagination-4       0.000 ± 0%
List/sqlite-overflows-page/all-4            0.000 ± 0%
List/sqlite-overflows-page/pagination-4     0.000 ± 0%
List/dqlite-tiny/all-4                      0.000 ± 0%
List/dqlite-tiny/pagination-4               0.000 ± 0%
List/dqlite-fits-in-page/all-4              0.000 ± 0%
List/dqlite-fits-in-page/pagination-4       0.000 ± 0%
List/dqlite-overflows-page/all-4            0.000 ± 0%
List/dqlite-overflows-page/pagination-4     0.000 ± 0%
Update/sqlite/1-workers-4                   4.245 ± 0%
Update/sqlite/4-workers-4                   4.314 ± 1%
Update/sqlite/16-workers-4                  4.336 ± 1%
Update/sqlite/64-workers-4                  4.375 ± 2%
Update/sqlite/128-workers-4                 4.423 ± 2%
Update/dqlite/1-workers-4                   4.205 ± 0%
Update/dqlite/4-workers-4                   4.209 ± 1%
Update/dqlite/16-workers-4                  4.239 ± 1%
Update/dqlite/64-workers-4                  4.360 ± 2%
Update/dqlite/128-workers-4                 4.419 ± 4%
geomean                                                ¹
¹ summaries must be >0 to compute geomean

                                        │     After      │
                                        │ sec-reading/op │
Compaction/sqlite-4                         27.80µ ± 39%
Compaction/dqlite-4                         29.90µ ± 73%
Create/sqlite/1-workers-4                   20.30µ ± 44%
Create/sqlite/4-workers-4                   19.60µ ± 26%
Create/sqlite/16-workers-4                  20.30µ ± 15%
Create/sqlite/64-workers-4                  25.50µ ± 39%
Create/sqlite/128-workers-4                 23.20µ ± 21%
Create/dqlite/1-workers-4                   10.00µ ± 51%
Create/dqlite/4-workers-4                   12.80µ ± 25%
Create/dqlite/16-workers-4                  12.60µ ± 60%
Create/dqlite/64-workers-4                  10.30µ ± 43%
Create/dqlite/128-workers-4                 12.40µ ± 79%
Delete/sqlite/1-workers-4                   32.10µ ± 19%
Delete/sqlite/4-workers-4                   35.30µ ± 23%
Delete/sqlite/16-workers-4                  32.10µ ± 31%
Delete/sqlite/64-workers-4                  36.00µ ± 26%
Delete/sqlite/128-workers-4                 34.90µ ± 34%
Delete/dqlite/1-workers-4                   14.50µ ± 39%
Delete/dqlite/4-workers-4                   17.10µ ± 34%
Delete/dqlite/16-workers-4                  15.20µ ± 37%
Delete/dqlite/64-workers-4                  15.70µ ± 45%
Delete/dqlite/128-workers-4                 15.30µ ± 56%
Get/sqlite-4                                64.40µ ±  8%
Get/dqlite-4                                33.20µ ± 11%
List/sqlite-tiny/all-4                      6.200µ ± 21%
List/sqlite-tiny/pagination-4               14.80µ ±  5%
List/sqlite-fits-in-page/all-4              12.80µ ± 25%
List/sqlite-fits-in-page/pagination-4       38.60µ ± 17%
List/sqlite-overflows-page/all-4            13.50µ ±  3%
List/sqlite-overflows-page/pagination-4     47.00µ ± 28%
List/dqlite-tiny/all-4                      4.100µ ± 20%
List/dqlite-tiny/pagination-4               8.900µ ± 13%
List/dqlite-fits-in-page/all-4              10.80µ ± 17%
List/dqlite-fits-in-page/pagination-4       25.30µ ±  8%
List/dqlite-overflows-page/all-4            34.70µ ±  3%
List/dqlite-overflows-page/pagination-4     67.00µ ±  6%
Update/sqlite/1-workers-4                   19.20µ ± 34%
Update/sqlite/4-workers-4                   21.80µ ± 17%
Update/sqlite/16-workers-4                  22.00µ ± 20%
Update/sqlite/64-workers-4                  24.00µ ± 27%
Update/sqlite/128-workers-4                 28.20µ ± 19%
Update/dqlite/1-workers-4                   10.40µ ± 31%
Update/dqlite/4-workers-4                   11.50µ ± 40%
Update/dqlite/16-workers-4                  11.70µ ± 21%
Update/dqlite/64-workers-4                  12.90µ ± 50%
Update/dqlite/128-workers-4                 11.30µ ± 61%
geomean                                     18.97µ

                                        │      After      │
                                        │ sec-writing/op  │
Compaction/sqlite-4                       9.200µ ±   7%
Compaction/dqlite-4                       6.600µ ±   9%
Create/sqlite/1-workers-4                 267.4µ ±   4%
Create/sqlite/4-workers-4                 1.407m ±  13%
Create/sqlite/16-workers-4                5.240m ±   7%
Create/sqlite/64-workers-4                18.58m ±  81%
Create/sqlite/128-workers-4               33.83m ±  60%
Create/dqlite/1-workers-4                 45.00µ ±  23%
Create/dqlite/4-workers-4                 49.70µ ±  13%
Create/dqlite/16-workers-4                49.00µ ±  21%
Create/dqlite/64-workers-4                41.30µ ±  20%
Create/dqlite/128-workers-4               42.40µ ±  26%
Delete/sqlite/1-workers-4                 237.2µ ±   9%
Delete/sqlite/4-workers-4                 1.378m ±  13%
Delete/sqlite/16-workers-4                5.315m ±  18%
Delete/sqlite/64-workers-4                18.70m ± 354%
Delete/sqlite/128-workers-4               35.08m ±  11%
Delete/dqlite/1-workers-4                 40.60µ ±  28%
Delete/dqlite/4-workers-4                 49.90µ ±  20%
Delete/dqlite/16-workers-4                45.90µ ±  26%
Delete/dqlite/64-workers-4                45.90µ ±  28%
Delete/dqlite/128-workers-4               49.30µ ±  32%
Get/sqlite-4                               0.000 ±   0%
Get/dqlite-4                               0.000 ±   0%
List/sqlite-tiny/all-4                     0.000 ±   0%
List/sqlite-tiny/pagination-4              0.000 ±   0%
List/sqlite-fits-in-page/all-4             0.000 ±   0%
List/sqlite-fits-in-page/pagination-4      0.000 ±   0%
List/sqlite-overflows-page/all-4           0.000 ±   0%
List/sqlite-overflows-page/pagination-4    0.000 ±   0%
List/dqlite-tiny/all-4                     0.000 ±   0%
List/dqlite-tiny/pagination-4              0.000 ±   0%
List/dqlite-fits-in-page/all-4             0.000 ±   0%
List/dqlite-fits-in-page/pagination-4      0.000 ±   0%
List/dqlite-overflows-page/all-4           0.000 ±   0%
List/dqlite-overflows-page/pagination-4    0.000 ±   0%
Update/sqlite/1-workers-4                 258.5µ ±   5%
Update/sqlite/4-workers-4                 1.398m ±   8%
Update/sqlite/16-workers-4                5.347m ±   8%
Update/sqlite/64-workers-4                18.41m ±   5%
Update/sqlite/128-workers-4               34.69m ±  31%
Update/dqlite/1-workers-4                 36.20µ ±  19%
Update/dqlite/4-workers-4                 39.50µ ±  13%
Update/dqlite/16-workers-4                40.70µ ±  33%
Update/dqlite/64-workers-4                47.20µ ±  25%
Update/dqlite/128-workers-4               44.20µ ±  27%
geomean                                                 ¹
¹ summaries must be >0 to compute geomean

                                        │     After     │
                                        │     B/op      │
Compaction/sqlite-4                       2.353Ki ± 49%
Compaction/dqlite-4                       5.180Ki ± 90%
Create/sqlite/1-workers-4                 20.50Ki ±  0%
Create/sqlite/4-workers-4                 20.49Ki ±  0%
Create/sqlite/16-workers-4                20.47Ki ±  0%
Create/sqlite/64-workers-4                20.52Ki ±  2%
Create/sqlite/128-workers-4               20.57Ki ±  2%
Create/dqlite/1-workers-4                 19.75Ki ±  0%
Create/dqlite/4-workers-4                 19.72Ki ±  0%
Create/dqlite/16-workers-4                19.73Ki ±  0%
Create/dqlite/64-workers-4                19.72Ki ±  1%
Create/dqlite/128-workers-4               19.74Ki ± 11%
Delete/sqlite/1-workers-4                 22.92Ki ±  0%
Delete/sqlite/4-workers-4                 22.92Ki ±  0%
Delete/sqlite/16-workers-4                22.89Ki ±  0%
Delete/sqlite/64-workers-4                22.92Ki ±  9%
Delete/sqlite/128-workers-4               23.02Ki ±  0%
Delete/dqlite/1-workers-4                 22.02Ki ±  0%
Delete/dqlite/4-workers-4                 21.99Ki ±  0%
Delete/dqlite/16-workers-4                21.99Ki ±  0%
Delete/dqlite/64-workers-4                21.99Ki ± 10%
Delete/dqlite/128-workers-4               22.01Ki ± 10%
Get/sqlite-4                              24.42Ki ±  0%
Get/dqlite-4                              23.25Ki ±  0%
List/sqlite-tiny/all-4                    1.142Ki ± 15%
List/sqlite-tiny/pagination-4             1.580Ki ±  8%
List/sqlite-fits-in-page/all-4            7.337Ki ± 22%
List/sqlite-fits-in-page/pagination-4     10.52Ki ± 14%
List/sqlite-overflows-page/all-4          20.54Ki ±  0%
List/sqlite-overflows-page/pagination-4   29.50Ki ± 42%
List/dqlite-tiny/all-4                    1.061Ki ±  6%
List/dqlite-tiny/pagination-4               981.0 ±  5%
List/dqlite-fits-in-page/all-4            6.031Ki ±  6%
List/dqlite-fits-in-page/pagination-4     5.426Ki ±  5%
List/dqlite-overflows-page/all-4          30.20Ki ±  8%
List/dqlite-overflows-page/pagination-4   29.27Ki ±  7%
Update/sqlite/1-workers-4                 22.28Ki ±  0%
Update/sqlite/4-workers-4                 22.29Ki ±  0%
Update/sqlite/16-workers-4                22.38Ki ±  0%
Update/sqlite/64-workers-4                22.63Ki ±  1%
Update/sqlite/128-workers-4               23.15Ki ±  3%
Update/dqlite/1-workers-4                 21.49Ki ±  0%
Update/dqlite/4-workers-4                 21.46Ki ±  0%
Update/dqlite/16-workers-4                21.69Ki ±  1%
Update/dqlite/64-workers-4                22.11Ki ±  2%
Update/dqlite/128-workers-4               22.55Ki ±  4%
geomean                                   14.37Ki

                                        │    After    │
                                        │  allocs/op  │
Compaction/sqlite-4                       50.00 ± 50%
Compaction/dqlite-4                       124.0 ± 91%
Create/sqlite/1-workers-4                 399.0 ±  0%
Create/sqlite/4-workers-4                 399.0 ±  0%
Create/sqlite/16-workers-4                399.0 ±  0%
Create/sqlite/64-workers-4                399.0 ±  1%
Create/sqlite/128-workers-4               400.0 ±  1%
Create/dqlite/1-workers-4                 369.0 ±  0%
Create/dqlite/4-workers-4                 369.0 ±  0%
Create/dqlite/16-workers-4                369.0 ±  0%
Create/dqlite/64-workers-4                368.0 ±  1%
Create/dqlite/128-workers-4               368.0 ± 14%
Delete/sqlite/1-workers-4                 431.0 ±  0%
Delete/sqlite/4-workers-4                 431.0 ±  0%
Delete/sqlite/16-workers-4                430.0 ±  0%
Delete/sqlite/64-workers-4                430.0 ±  0%
Delete/sqlite/128-workers-4               431.0 ±  0%
Delete/dqlite/1-workers-4                 399.0 ±  0%
Delete/dqlite/4-workers-4                 399.0 ±  0%
Delete/dqlite/16-workers-4                399.0 ±  0%
Delete/dqlite/64-workers-4                399.0 ± 13%
Delete/dqlite/128-workers-4               399.0 ± 13%
Get/sqlite-4                              445.0 ±  0%
Get/dqlite-4                              412.0 ±  0%
List/sqlite-tiny/all-4                    15.00 ± 20%
List/sqlite-tiny/pagination-4             25.00 ±  8%
List/sqlite-fits-in-page/all-4            20.00 ± 30%
List/sqlite-fits-in-page/pagination-4     38.00 ± 18%
List/sqlite-overflows-page/all-4          9.000 ±  0%
List/sqlite-overflows-page/pagination-4   18.00 ± 50%
List/dqlite-tiny/all-4                    15.00 ±  7%
List/dqlite-tiny/pagination-4             14.00 ±  7%
List/dqlite-fits-in-page/all-4            19.00 ±  5%
List/dqlite-fits-in-page/pagination-4     19.00 ±  0%
List/dqlite-overflows-page/all-4          33.00 ±  3%
List/dqlite-overflows-page/pagination-4   33.00 ±  3%
Update/sqlite/1-workers-4                 412.0 ±  0%
Update/sqlite/4-workers-4                 412.0 ±  0%
Update/sqlite/16-workers-4                413.0 ±  0%
Update/sqlite/64-workers-4                418.0 ±  1%
Update/sqlite/128-workers-4               427.0 ±  3%
Update/dqlite/1-workers-4                 381.0 ±  0%
Update/dqlite/4-workers-4                 381.0 ±  0%
Update/dqlite/16-workers-4                385.0 ±  1%
Update/dqlite/64-workers-4                392.0 ±  2%
Update/dqlite/128-workers-4               397.0 ±  5%
geomean                                   170.7

                                        │         After         │
                                        │ network-bytes-read/op │
Compaction/dqlite-4                               1.871Ki ± 92%
Create/dqlite/1-workers-4                           240.9 ±  0%
Create/dqlite/4-workers-4                           241.7 ±  0%
Create/dqlite/16-workers-4                          242.0 ±  0%
Create/dqlite/64-workers-4                          242.4 ±  4%
Create/dqlite/128-workers-4                         242.4 ± 35%
Delete/dqlite/1-workers-4                           717.8 ±  0%
Delete/dqlite/4-workers-4                           719.2 ±  0%
Delete/dqlite/16-workers-4                          719.3 ±  0%
Delete/dqlite/64-workers-4                          718.5 ± 12%
Delete/dqlite/128-workers-4                         719.2 ± 12%
Get/dqlite-4                                      1.133Ki ±  0%
List/dqlite-tiny/all-4                              213.2 ±  3%
List/dqlite-tiny/pagination-4                       198.0 ±  3%
List/dqlite-fits-in-page/all-4                    1.341Ki ±  4%
List/dqlite-fits-in-page/pagination-4             1.306Ki ±  3%
List/dqlite-overflows-page/all-4                  6.971Ki ±  2%
List/dqlite-overflows-page/pagination-4           6.849Ki ±  4%
Update/dqlite/1-workers-4                           262.8 ±  0%
Update/dqlite/4-workers-4                           263.2 ±  0%
Update/dqlite/16-workers-4                          265.3 ±  1%
Update/dqlite/64-workers-4                          270.4 ±  2%
Update/dqlite/128-workers-4                         273.9 ±  4%
geomean                                             563.5

                                        │          After           │
                                        │ network-bytes-written/op │
Compaction/dqlite-4                                    4.532 ± 14%
Create/dqlite/1-workers-4                              115.1 ±  0%
Create/dqlite/4-workers-4                              116.5 ±  2%
Create/dqlite/16-workers-4                             117.1 ±  1%
Create/dqlite/64-workers-4                             117.7 ±  2%
Create/dqlite/128-workers-4                            117.7 ± 23%
Delete/dqlite/1-workers-4                              82.31 ±  1%
Delete/dqlite/4-workers-4                              83.08 ±  0%
Delete/dqlite/16-workers-4                             83.09 ±  1%
Delete/dqlite/64-workers-4                             82.80 ± 35%
Delete/dqlite/128-workers-4                            83.09 ± 34%
Get/dqlite-4                                           80.50 ±  0%
List/dqlite-tiny/all-4                               0.04705 ±  5%
List/dqlite-tiny/pagination-4                        0.04865 ± 11%
List/dqlite-fits-in-page/all-4                       0.06089 ±  6%
List/dqlite-fits-in-page/pagination-4                0.08282 ± 14%
List/dqlite-overflows-page/all-4                     0.07805 ±  3%
List/dqlite-overflows-page/pagination-4               0.1205 ± 10%
Update/dqlite/1-workers-4                              136.6 ±  0%
Update/dqlite/4-workers-4                              136.7 ±  0%
Update/dqlite/16-workers-4                             137.7 ±  1%
Update/dqlite/64-workers-4                             140.9 ±  2%
Update/dqlite/128-workers-4                            143.3 ±  4%
geomean                                                13.84

Copy link

@addyess addyess left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most of my comments are around sourcing of various files. I don't quite know how to review more than the logic bits of the rscript which looks valid enough

.github/workflows/e2e.yml Outdated Show resolved Hide resolved
.github/workflows/performance.yaml Outdated Show resolved Hide resolved
test/performance/lxd-profile.yaml Outdated Show resolved Hide resolved
@@ -0,0 +1,73 @@
#!/usr/bin/env Rscript
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not familiar with r-script. Did you pull this from somewhere else? Should we be vendoring this into the project or can it be sourced from elsewhere?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, this is my magic, I used to work for R :)

test/performance/tests/test_performance.py Outdated Show resolved Hide resolved
Base automatically changed from KU-1874/performance-workload to master November 14, 2024 09:53
@louiseschmidtgen louiseschmidtgen marked this pull request as ready for review November 14, 2024 11:50
@louiseschmidtgen louiseschmidtgen requested a review from a team as a code owner November 14, 2024 11:50
cd test/performance
mkdir -p ./results/base-code
sg lxd -c 'tox -e performance'
- name: Run Performance test for v1.1.11 snap
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of curiosity, why are you specifically testing v1.1.11? What's special about this version?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The v1.1.11 is the tag before we introduced the compaction changes (v1.2.0).
So we can compare where we are now with v1.11.1:

image

Copy link

@addyess addyess left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@louiseschmidtgen louiseschmidtgen merged commit 5726162 into master Nov 14, 2024
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants