Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[profiling] Use the new FFI macros and types #797

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

danielsn
Copy link
Contributor

@danielsn danielsn commented Dec 10, 2024

What does this PR do?

Uses the macros introduced in #752 in the profiler-ffi

Motivation

DRY: Don't Repeat Yourself ;)

Additional Notes

This has breaking FFI changes (changes some types and type names) but the upgrade should be pretty mechanical clientside.

How to test the change?

This is a refactor change

@danielsn danielsn requested review from a team as code owners December 10, 2024 21:47
@github-actions github-actions bot added profiling Relates to the profiling* modules. common labels Dec 10, 2024
@danielsn danielsn force-pushed the dsn/profiler-use-macros branch from 31cd819 to 1374f98 Compare December 10, 2024 21:51
@pr-commenter
Copy link

pr-commenter bot commented Dec 10, 2024

Benchmarks

Comparison

Benchmark execution time: 2024-12-10 22:50:54

Comparing candidate commit ef10d90 in PR branch dsn/profiler-use-macros with baseline commit 373b778 in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 51 metrics, 2 unstable metrics.

Candidate

Candidate benchmark details

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz ef10d90 1733870400 dsn/profiler-use-macros
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching deserializing traces from msgpack to their internal representation execution_time 58.754ms 58.963ms ± 0.129ms 58.946ms ± 0.051ms 58.990ms 59.174ms 59.413ms 59.805ms 1.46% 2.823 12.790 0.22% 0.009ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching deserializing traces from msgpack to their internal representation execution_time [58.945ms; 58.981ms] or [-0.030%; +0.030%] None None None

Group 2

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz ef10d90 1733870400 dsn/profiler-use-macros
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time 188.993µs 191.912µs ± 0.805µs 191.942µs ± 0.485µs 192.451µs 193.014µs 193.857µs 194.121µs 1.14% -0.375 0.622 0.42% 0.057µs 1 200
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput 5151412.904op/s 5210805.146op/s ± 21902.891op/s 5209914.304op/s ± 13175.941op/s 5221409.041op/s 5249985.605op/s 5261015.200op/s 5291210.103op/s 1.56% 0.406 0.654 0.42% 1548.768op/s 1 200
normalization/normalize_name/normalize_name/bad-name execution_time 18.110µs 18.208µs ± 0.050µs 18.210µs ± 0.034µs 18.241µs 18.279µs 18.302µs 18.473µs 1.45% 0.525 2.517 0.28% 0.004µs 1 200
normalization/normalize_name/normalize_name/bad-name throughput 54133674.694op/s 54922468.857op/s ± 152069.279op/s 54916073.017op/s ± 103417.412op/s 55030123.297op/s 55163444.427op/s 55197045.861op/s 55219546.420op/s 0.55% -0.491 2.342 0.28% 10752.922op/s 1 200
normalization/normalize_name/normalize_name/good execution_time 11.034µs 11.090µs ± 0.025µs 11.084µs ± 0.016µs 11.111µs 11.128µs 11.141µs 11.148µs 0.58% 0.087 -0.614 0.23% 0.002µs 1 200
normalization/normalize_name/normalize_name/good throughput 89700365.739op/s 90175319.214op/s ± 203552.652op/s 90218960.716op/s ± 130004.132op/s 90311259.892op/s 90527483.556op/s 90593823.780op/s 90629735.050op/s 0.46% -0.077 -0.612 0.23% 14393.346op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time [191.801µs; 192.024µs] or [-0.058%; +0.058%] None None None
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput [5207769.616op/s; 5213840.676op/s] or [-0.058%; +0.058%] None None None
normalization/normalize_name/normalize_name/bad-name execution_time [18.201µs; 18.215µs] or [-0.038%; +0.038%] None None None
normalization/normalize_name/normalize_name/bad-name throughput [54901393.518op/s; 54943544.197op/s] or [-0.038%; +0.038%] None None None
normalization/normalize_name/normalize_name/good execution_time [11.086µs; 11.093µs] or [-0.031%; +0.031%] None None None
normalization/normalize_name/normalize_name/good throughput [90147108.774op/s; 90203529.654op/s] or [-0.031%; +0.031%] None None None

Group 3

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz ef10d90 1733870400 dsn/profiler-use-macros
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching string interning on wordpress profile execution_time 139.775µs 140.794µs ± 0.353µs 140.773µs ± 0.208µs 140.973µs 141.327µs 141.871µs 142.709µs 1.38% 1.357 6.106 0.25% 0.025µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching string interning on wordpress profile execution_time [140.745µs; 140.843µs] or [-0.035%; +0.035%] None None None

Group 4

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz ef10d90 1733870400 dsn/profiler-use-macros
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
two way interface execution_time 18.597µs 23.903µs ± 11.781µs 19.042µs ± 0.243µs 19.246µs 38.391µs 47.592µs 125.703µs 560.15% 4.709 32.634 49.17% 0.833µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
two way interface execution_time [22.270µs; 25.535µs] or [-6.831%; +6.831%] None None None

Group 5

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz ef10d90 1733870400 dsn/profiler-use-macros
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
concentrator/add_spans_to_concentrator execution_time 6.764ms 6.779ms ± 0.010ms 6.778ms ± 0.004ms 6.782ms 6.789ms 6.822ms 6.859ms 1.21% 4.295 27.332 0.15% 0.001ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
concentrator/add_spans_to_concentrator execution_time [6.778ms; 6.780ms] or [-0.021%; +0.021%] None None None

Group 6

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz ef10d90 1733870400 dsn/profiler-use-macros
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
tags/replace_trace_tags execution_time 2.718µs 2.756µs ± 0.018µs 2.751µs ± 0.009µs 2.767µs 2.789µs 2.802µs 2.804µs 1.90% 0.605 -0.314 0.66% 0.001µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
tags/replace_trace_tags execution_time [2.753µs; 2.758µs] or [-0.091%; +0.091%] None None None

Group 7

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz ef10d90 1733870400 dsn/profiler-use-macros
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
write only interface execution_time 1.411µs 3.231µs ± 1.434µs 3.078µs ± 0.023µs 3.097µs 3.133µs 14.188µs 15.206µs 394.00% 7.661 58.484 44.27% 0.101µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
write only interface execution_time [3.032µs; 3.429µs] or [-6.150%; +6.150%] None None None

Group 8

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz ef10d90 1733870400 dsn/profiler-use-macros
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
sql/obfuscate_sql_string execution_time 72.427µs 72.684µs ± 0.631µs 72.614µs ± 0.063µs 72.695µs 72.808µs 73.025µs 81.072µs 11.65% 12.155 155.879 0.87% 0.045µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
sql/obfuscate_sql_string execution_time [72.596µs; 72.771µs] or [-0.120%; +0.120%] None None None

Group 9

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz ef10d90 1733870400 dsn/profiler-use-macros
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_trace/test_trace execution_time 273.039ns 284.013ns ± 12.839ns 276.988ns ± 3.485ns 290.498ns 314.542ns 320.416ns 321.281ns 15.99% 1.451 1.133 4.51% 0.908ns 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_trace/test_trace execution_time [282.234ns; 285.793ns] or [-0.627%; +0.627%] None None None

Group 10

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz ef10d90 1733870400 dsn/profiler-use-macros
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
credit_card/is_card_number/ execution_time 4.613µs 4.630µs ± 0.007µs 4.630µs ± 0.004µs 4.634µs 4.639µs 4.644µs 4.690µs 1.31% 3.379 28.218 0.15% 0.000µs 1 200
credit_card/is_card_number/ throughput 213211495.938op/s 215973243.647op/s ± 316925.619op/s 216005044.183op/s ± 180328.602op/s 216155087.124op/s 216323724.164op/s 216530306.167op/s 216795479.900op/s 0.37% -3.297 27.320 0.15% 22410.025op/s 1 200
credit_card/is_card_number/ 3782-8224-6310-005 execution_time 95.145µs 95.666µs ± 0.498µs 95.624µs ± 0.178µs 95.796µs 96.208µs 96.521µs 101.308µs 5.94% 7.401 80.845 0.52% 0.035µs 1 200
credit_card/is_card_number/ 3782-8224-6310-005 throughput 9870852.585op/s 10453252.655op/s ± 52411.609op/s 10457601.268op/s ± 19439.878op/s 10477619.299op/s 10500721.981op/s 10507541.325op/s 10510250.647op/s 0.50% -6.974 74.395 0.50% 3706.060op/s 1 200
credit_card/is_card_number/ 378282246310005 execution_time 85.507µs 86.173µs ± 0.446µs 86.134µs ± 0.210µs 86.348µs 86.659µs 87.092µs 90.679µs 5.28% 5.238 50.322 0.52% 0.032µs 1 200
credit_card/is_card_number/ 378282246310005 throughput 11027854.475op/s 11604807.081op/s ± 58546.776op/s 11609872.559op/s ± 28329.485op/s 11636897.211op/s 11676752.520op/s 11689507.604op/s 11694903.361op/s 0.73% -4.866 45.441 0.50% 4139.882op/s 1 200
credit_card/is_card_number/37828224631 execution_time 4.610µs 4.628µs ± 0.008µs 4.627µs ± 0.003µs 4.631µs 4.637µs 4.643µs 4.697µs 1.50% 3.852 33.699 0.16% 0.001µs 1 200
credit_card/is_card_number/37828224631 throughput 212912256.406op/s 216079526.047op/s ± 348982.202op/s 216109422.395op/s ± 158795.442op/s 216251938.628op/s 216495976.646op/s 216784957.395op/s 216929168.360op/s 0.38% -3.751 32.553 0.16% 24676.768op/s 1 200
credit_card/is_card_number/378282246310005 execution_time 82.207µs 82.823µs ± 0.309µs 82.819µs ± 0.210µs 83.009µs 83.410µs 83.603µs 83.738µs 1.11% 0.466 0.003 0.37% 0.022µs 1 200
credit_card/is_card_number/378282246310005 throughput 11942057.925op/s 12074075.811op/s ± 44993.045op/s 12074454.078op/s ± 30572.625op/s 12106476.664op/s 12142107.441op/s 12152298.987op/s 12164470.477op/s 0.75% -0.446 -0.024 0.37% 3181.489op/s 1 200
credit_card/is_card_number/37828224631000521389798 execution_time 58.991µs 59.095µs ± 0.063µs 59.076µs ± 0.024µs 59.113µs 59.225µs 59.296µs 59.312µs 0.40% 1.376 1.589 0.11% 0.004µs 1 200
credit_card/is_card_number/37828224631000521389798 throughput 16860069.850op/s 16921865.413op/s ± 18063.203op/s 16927393.332op/s ± 6837.502op/s 16932884.818op/s 16941396.069op/s 16949772.232op/s 16951609.934op/s 0.14% -1.371 1.572 0.11% 1277.261op/s 1 200
credit_card/is_card_number/x371413321323331 execution_time 6.832µs 6.844µs ± 0.004µs 6.843µs ± 0.002µs 6.846µs 6.851µs 6.854µs 6.859µs 0.22% 0.364 0.901 0.06% 0.000µs 1 200
credit_card/is_card_number/x371413321323331 throughput 145802614.973op/s 146122075.130op/s ± 90197.536op/s 146127842.078op/s ± 49199.224op/s 146167729.821op/s 146279771.479op/s 146325604.146op/s 146363107.046op/s 0.16% -0.359 0.895 0.06% 6377.929op/s 1 200
credit_card/is_card_number_no_luhn/ execution_time 4.612µs 4.628µs ± 0.006µs 4.627µs ± 0.004µs 4.632µs 4.637µs 4.641µs 4.641µs 0.30% 0.053 -0.339 0.12% 0.000µs 1 200
credit_card/is_card_number_no_luhn/ throughput 215452973.176op/s 216088884.657op/s ± 266867.524op/s 216106645.285op/s ± 176799.378op/s 216265134.650op/s 216521878.856op/s 216603501.409op/s 216844644.509op/s 0.34% -0.047 -0.339 0.12% 18870.384op/s 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time 74.993µs 75.451µs ± 0.098µs 75.449µs ± 0.063µs 75.518µs 75.594µs 75.666µs 75.691µs 0.32% -0.489 1.822 0.13% 0.007µs 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput 13211555.243op/s 13253729.626op/s ± 17202.370op/s 13254021.138op/s ± 11105.388op/s 13264073.599op/s 13280888.669op/s 13288386.347op/s 13334505.700op/s 0.61% 0.503 1.868 0.13% 1216.391op/s 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time 65.904µs 66.093µs ± 0.096µs 66.084µs ± 0.064µs 66.152µs 66.255µs 66.392µs 66.475µs 0.59% 0.776 1.020 0.14% 0.007µs 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 throughput 15043302.939op/s 15130208.705op/s ± 21933.747op/s 15132266.464op/s ± 14618.337op/s 15145082.794op/s 15161231.292op/s 15167263.843op/s 15173557.042op/s 0.27% -0.766 0.989 0.14% 1550.950op/s 1 200
credit_card/is_card_number_no_luhn/37828224631 execution_time 4.620µs 4.631µs ± 0.006µs 4.631µs ± 0.004µs 4.634µs 4.640µs 4.643µs 4.660µs 0.62% 0.737 2.348 0.12% 0.000µs 1 200
credit_card/is_card_number_no_luhn/37828224631 throughput 214612082.203op/s 215959188.803op/s ± 266386.582op/s 215944728.953op/s ± 168456.579op/s 216154842.529op/s 216380823.762op/s 216460544.051op/s 216465031.828op/s 0.24% -0.723 2.283 0.12% 18836.376op/s 1 200
credit_card/is_card_number_no_luhn/378282246310005 execution_time 62.497µs 63.027µs ± 0.162µs 63.002µs ± 0.105µs 63.143µs 63.298µs 63.412µs 63.426µs 0.67% 0.067 0.320 0.26% 0.011µs 1 200
credit_card/is_card_number_no_luhn/378282246310005 throughput 15766332.378op/s 15866392.544op/s ± 40721.592op/s 15872411.837op/s ± 26467.342op/s 15890253.038op/s 15924121.829op/s 15966447.015op/s 16000760.086op/s 0.81% -0.049 0.337 0.26% 2879.451op/s 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time 58.997µs 59.087µs ± 0.040µs 59.084µs ± 0.016µs 59.098µs 59.158µs 59.243µs 59.272µs 0.32% 1.426 4.660 0.07% 0.003µs 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput 16871444.653op/s 16924148.150op/s ± 11512.711op/s 16925148.531op/s ± 4707.240op/s 16930433.273op/s 16937966.561op/s 16948180.920op/s 16950080.318op/s 0.15% -1.416 4.623 0.07% 814.072op/s 1 200
credit_card/is_card_number_no_luhn/x371413321323331 execution_time 6.833µs 6.843µs ± 0.004µs 6.843µs ± 0.003µs 6.846µs 6.849µs 6.853µs 6.854µs 0.16% -0.021 0.101 0.06% 0.000µs 1 200
credit_card/is_card_number_no_luhn/x371413321323331 throughput 145901529.242op/s 146134230.952op/s ± 84819.386op/s 146129090.860op/s ± 53685.876op/s 146183466.650op/s 146281871.770op/s 146329552.286op/s 146347591.546op/s 0.15% 0.025 0.100 0.06% 5997.636op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
credit_card/is_card_number/ execution_time [4.629µs; 4.631µs] or [-0.020%; +0.020%] None None None
credit_card/is_card_number/ throughput [215929320.804op/s; 216017166.489op/s] or [-0.020%; +0.020%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 execution_time [95.598µs; 95.735µs] or [-0.072%; +0.072%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 throughput [10445988.910op/s; 10460516.400op/s] or [-0.069%; +0.069%] None None None
credit_card/is_card_number/ 378282246310005 execution_time [86.112µs; 86.235µs] or [-0.072%; +0.072%] None None None
credit_card/is_card_number/ 378282246310005 throughput [11596693.061op/s; 11612921.101op/s] or [-0.070%; +0.070%] None None None
credit_card/is_card_number/37828224631 execution_time [4.627µs; 4.629µs] or [-0.023%; +0.023%] None None None
credit_card/is_card_number/37828224631 throughput [216031160.470op/s; 216127891.624op/s] or [-0.022%; +0.022%] None None None
credit_card/is_card_number/378282246310005 execution_time [82.780µs; 82.866µs] or [-0.052%; +0.052%] None None None
credit_card/is_card_number/378282246310005 throughput [12067840.208op/s; 12080311.415op/s] or [-0.052%; +0.052%] None None None
credit_card/is_card_number/37828224631000521389798 execution_time [59.086µs; 59.104µs] or [-0.015%; +0.015%] None None None
credit_card/is_card_number/37828224631000521389798 throughput [16919362.026op/s; 16924368.799op/s] or [-0.015%; +0.015%] None None None
credit_card/is_card_number/x371413321323331 execution_time [6.843µs; 6.844µs] or [-0.009%; +0.009%] None None None
credit_card/is_card_number/x371413321323331 throughput [146109574.619op/s; 146134575.641op/s] or [-0.009%; +0.009%] None None None
credit_card/is_card_number_no_luhn/ execution_time [4.627µs; 4.629µs] or [-0.017%; +0.017%] None None None
credit_card/is_card_number_no_luhn/ throughput [216051899.385op/s; 216125869.929op/s] or [-0.017%; +0.017%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time [75.437µs; 75.464µs] or [-0.018%; +0.018%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput [13251345.543op/s; 13256113.709op/s] or [-0.018%; +0.018%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time [66.080µs; 66.106µs] or [-0.020%; +0.020%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 throughput [15127168.899op/s; 15133248.511op/s] or [-0.020%; +0.020%] None None None
credit_card/is_card_number_no_luhn/37828224631 execution_time [4.630µs; 4.631µs] or [-0.017%; +0.017%] None None None
credit_card/is_card_number_no_luhn/37828224631 throughput [215922270.185op/s; 215996107.422op/s] or [-0.017%; +0.017%] None None None
credit_card/is_card_number_no_luhn/378282246310005 execution_time [63.004µs; 63.049µs] or [-0.036%; +0.036%] None None None
credit_card/is_card_number_no_luhn/378282246310005 throughput [15860748.923op/s; 15872036.165op/s] or [-0.036%; +0.036%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time [59.082µs; 59.093µs] or [-0.009%; +0.009%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput [16922552.599op/s; 16925743.701op/s] or [-0.009%; +0.009%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 execution_time [6.842µs; 6.844µs] or [-0.008%; +0.008%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 throughput [146122475.801op/s; 146145986.103op/s] or [-0.008%; +0.008%] None None None

Group 11

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz ef10d90 1733870400 dsn/profiler-use-macros
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
redis/obfuscate_redis_string execution_time 38.071µs 38.571µs ± 0.848µs 38.180µs ± 0.046µs 38.320µs 40.320µs 40.375µs 42.442µs 11.16% 1.872 2.278 2.19% 0.060µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
redis/obfuscate_redis_string execution_time [38.453µs; 38.688µs] or [-0.305%; +0.305%] None None None

Group 12

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz ef10d90 1733870400 dsn/profiler-use-macros
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time 626.099µs 627.013µs ± 0.591µs 626.985µs ± 0.219µs 627.206µs 627.551µs 627.932µs 633.798µs 1.09% 7.589 84.858 0.09% 0.042µs 1 200
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput 1577788.880op/s 1594864.261op/s ± 1493.651op/s 1594934.216op/s ± 557.108op/s 1595484.414op/s 1596330.465op/s 1596737.566op/s 1597192.410op/s 0.14% -7.506 83.614 0.09% 105.617op/s 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time 463.470µs 464.475µs ± 0.368µs 464.485µs ± 0.237µs 464.705µs 465.090µs 465.345µs 465.393µs 0.20% 0.021 -0.257 0.08% 0.026µs 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput 2148719.674op/s 2152971.897op/s ± 1706.756op/s 2152920.593op/s ± 1098.740op/s 2154130.290op/s 2155518.249op/s 2156673.756op/s 2157636.711op/s 0.22% -0.017 -0.257 0.08% 120.686op/s 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time 179.284µs 179.945µs ± 0.645µs 179.897µs ± 0.263µs 180.137µs 180.425µs 180.798µs 185.634µs 3.19% 6.400 52.866 0.36% 0.046µs 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput 5386952.695op/s 5557328.337op/s ± 19488.158op/s 5558733.387op/s ± 8152.064op/s 5568443.713op/s 5573083.924op/s 5574995.116op/s 5577735.747op/s 0.34% -6.251 51.189 0.35% 1378.021op/s 1 200
normalization/normalize_service/normalize_service/[empty string] execution_time 47.486µs 47.651µs ± 0.066µs 47.651µs ± 0.034µs 47.682µs 47.753µs 47.795µs 48.086µs 0.91% 1.379 8.639 0.14% 0.005µs 1 200
normalization/normalize_service/normalize_service/[empty string] throughput 20796280.540op/s 20986156.988op/s ± 29008.104op/s 20986089.188op/s ± 14892.700op/s 21001569.411op/s 21033287.372op/s 21047313.997op/s 21058873.744op/s 0.35% -1.343 8.392 0.14% 2051.183op/s 1 200
normalization/normalize_service/normalize_service/test_ASCII execution_time 49.677µs 49.837µs ± 0.112µs 49.832µs ± 0.046µs 49.873µs 49.962µs 49.995µs 51.093µs 2.53% 7.030 76.645 0.22% 0.008µs 1 200
normalization/normalize_service/normalize_service/test_ASCII throughput 19572103.744op/s 20065509.717op/s ± 44490.835op/s 20067365.680op/s ± 18322.456op/s 20087695.643op/s 20115903.156op/s 20125841.274op/s 20130096.601op/s 0.31% -6.835 73.791 0.22% 3145.977op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time [626.931µs; 627.095µs] or [-0.013%; +0.013%] None None None
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput [1594657.255op/s; 1595071.267op/s] or [-0.013%; +0.013%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time [464.424µs; 464.526µs] or [-0.011%; +0.011%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput [2152735.357op/s; 2153208.437op/s] or [-0.011%; +0.011%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time [179.855µs; 180.034µs] or [-0.050%; +0.050%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput [5554627.466op/s; 5560029.209op/s] or [-0.049%; +0.049%] None None None
normalization/normalize_service/normalize_service/[empty string] execution_time [47.641µs; 47.660µs] or [-0.019%; +0.019%] None None None
normalization/normalize_service/normalize_service/[empty string] throughput [20982136.744op/s; 20990177.232op/s] or [-0.019%; +0.019%] None None None
normalization/normalize_service/normalize_service/test_ASCII execution_time [49.821µs; 49.853µs] or [-0.031%; +0.031%] None None None
normalization/normalize_service/normalize_service/test_ASCII throughput [20059343.715op/s; 20071675.719op/s] or [-0.031%; +0.031%] None None None

Baseline

Omitted due to size.

@codecov-commenter
Copy link

codecov-commenter commented Dec 10, 2024

Codecov Report

Attention: Patch coverage is 61.53846% with 75 lines in your changes missing coverage. Please review.

Project coverage is 71.04%. Comparing base (373b778) to head (ef10d90).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #797      +/-   ##
==========================================
+ Coverage   70.95%   71.04%   +0.09%     
==========================================
  Files         313      314       +1     
  Lines       45747    45665      -82     
==========================================
- Hits        32458    32441      -17     
+ Misses      13289    13224      -65     
Components Coverage Δ
crashtracker 38.27% <ø> (ø)
crashtracker-ffi 5.71% <ø> (ø)
datadog-alloc 98.73% <ø> (ø)
data-pipeline 92.06% <ø> (ø)
data-pipeline-ffi 90.54% <ø> (ø)
ddcommon 83.69% <77.77%> (+1.38%) ⬆️
ddcommon-ffi 69.58% <77.77%> (+4.05%) ⬆️
ddtelemetry 59.51% <ø> (ø)
ddtelemetry-ffi 22.46% <ø> (ø)
dogstatsd 89.45% <ø> (ø)
dogstatsd-client 79.77% <ø> (ø)
ipc 82.76% <ø> (ø)
profiling 84.81% <59.88%> (+0.49%) ⬆️
profiling-ffi 79.49% <59.88%> (+1.93%) ⬆️
serverless 0.00% <ø> (ø)
sidecar 40.56% <ø> (ø)
sidecar-ffi 1.18% <ø> (ø)
spawn-worker 54.37% <ø> (ø)
tinybytes 93.60% <ø> (ø)
trace-mini-agent 72.38% <ø> (ø)
trace-normalization 98.23% <ø> (ø)
trace-obfuscation 95.77% <ø> (ø)
trace-protobuf 77.67% <ø> (ø)
trace-utils 93.52% <ø> (ø)

@danielsn danielsn force-pushed the dsn/profiler-use-macros branch from 6680602 to 4a92b26 Compare December 10, 2024 22:09
@danielsn danielsn force-pushed the dsn/profiler-use-macros branch from 2ffb63d to ef10d90 Compare December 10, 2024 22:40
Copy link
Contributor

@morrisonlevi morrisonlevi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This has breaking FFI changes, which I didn't expect with a "refactor" title, but that's okay. Just wanted to call a bit of attention there, and we should write something about in this PR description.

I think I need to open this up in an editor to see some of it, diff-view isn't great sometimes.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't care about the C++ changes, if someone thinks it's better, it's fine to me.

@danielsn danielsn changed the title [profiling] Refactor: use the new FFI macros and types [profiling] Use the new FFI macros and types Dec 10, 2024
@danielsn
Copy link
Contributor Author

This has breaking FFI changes, which I didn't expect with a "refactor" title, but that's okay. Just wanted to call a bit of attention there, and we should write something about in this PR description.

I think I need to open this up in an editor to see some of it, diff-view isn't great sometimes.

Updated the PR description

Comment on lines +17 to +18
ddog_prof_Result_HandleProfile new_result = ddog_prof_Profile_new(sample_types, &period, NULL);
if (new_result.tag != DDOG_PROF_RESULT_HANDLE_PROFILE_OK_HANDLE_PROFILE) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't write C API these days, but this looks worse. Can we use renames and/or aliases to improve this? Examples:

  • ddog_prof_Profile_NewResult aliased to ddog_prof_Result_HandleProfile
  • ddog_prof_Profile aliased to ddog_prof_Handle_Profile

Not sure what to do about the enum. Maybe @ivoanjo has thoughts?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm yeah I think Levi has a good point here, the "handle" change does look a bit like libdatadog internal implementation details being leaked across the FFI.

E.g. looking at it from the pure API client side, I'm not sure it's relevant for client to care what a ddog_prof_Profile actually is: is it a profile? a handle? something else?

I also know that (at least in the past) some of our types were a bit less what we wanted because of cbindgen limitations, and I think it's still a reasonable trade-off to have the types uglier if the rust code becomes less error-prone/etc. But ideally having both (clear naming for ffi and better rust types) would be great.

ddog_prof_Profile aliased to ddog_prof_Handle_Profile

TL;DR: 👍 on this one :)


On the Result change...

We've been using the convention of calling Module_operation returning Module_OperationResult since this PR. (And we did explore a few more options at the time)

One advantage of this new convention is that it may result (no pun intended) in fewer ...Result types, because they are now named by their payload's type, rather than the operation's type.

I still think I slightly favor more the current convention, but I think both seem quite reasonable. Here's a few examples for a comparison:

Current convention with this PR's convention Notes
ddog_prof_Profile_NewResult ddog_prof_Result_HandleProfile
ddog_prof_Exporter_NewResult ddog_prof_Result_Exporter
ddog_prof_Exporter_Request_BuildResult ddog_prof_Result_Request? ddog_prof_Result_Exporter_Request? How to handle the deeper namespace?
ddog_prof_Exporter_SendResult ddog_prof_Result_HttpStatus ? ddog_Result_HttpStatus ? How to handle when the type in the result is not named after the operation?
ddog_prof_Profile_Result ddog_prof_ResultBool? ddog_ResultBool? (Similar to above)
ddog_prof_Profile_SerializeResult ddog_prof_Result_EncodedProfile

(There's also a few more results in ddcommon-ffi)

Not wanting to discourage changes (I think one of the key ingredients of libdatadog's massive success so far has been having "no sacred cows", and instead always striving to leave things better, even if in the very short term it causes a bit of migration pain) I'll suggest that if we plan to introduce the new convention, I don't think we should mix both -- I think we should rip-off the band-aid and convert everything to the new convention.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like these being moved into their own file 👍🏻

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
common profiling Relates to the profiling* modules.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants