-
-
Notifications
You must be signed in to change notification settings - Fork 435
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Refactor benchmarks #2722
Refactor benchmarks #2722
Conversation
Signed-off-by: Asish Kumar <[email protected]>
Signed-off-by: Asish Kumar <[email protected]>
Signed-off-by: Asish Kumar <[email protected]>
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #2722 +/- ##
==========================================
- Coverage 36.61% 36.58% -0.04%
==========================================
Files 186 186
Lines 14837 14859 +22
==========================================
+ Hits 5433 5436 +3
- Misses 9404 9423 +19 ☔ View full report in Codecov by Sentry. |
}, | ||
{ | ||
"nu": 0, | ||
"temperature": 1, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These aren't physical, I don't think they are worth parametrizing
}, | ||
{ | ||
"nu": 1, | ||
"temperature": 1, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same here
"Parameters": [ | ||
{ | ||
"nu": 1e14, | ||
"temperature": 1e4, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is okay
Signed-off-by: Asish Kumar <[email protected]>
Signed-off-by: Asish Kumar <[email protected]>
Signed-off-by: Asish Kumar <[email protected]>
*beep* *bop* Significantly changed benchmarks: All benchmarks: All benchmarks:
| Change | Before [fea0ff01] | After [fea0ff01] | Ratio | Benchmark (Parameter) |
|----------|----------------------|---------------------|---------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| | 38.2±0.1s | 38.2±0.1s | 1 | run_tardis.BenchmarkRunTardis.time_run_tardis |
| | 27.3±10μs | 27.3±10μs | 1 | transport_geometry_calculate_distances.BenchmarkTransportGeometryCalculateDistances.time_calculate_distance_boundary |
| | 44.0±4μs | 44.0±4μs | 1 | transport_geometry_calculate_distances.BenchmarkTransportGeometryCalculateDistances.time_calculate_distance_line({'packet': {'nu_line': 0.1, 'is_last_line': True}, 'enable_full_relativity': True}) |
| | 43.6±4μs | 43.6±4μs | 1 | transport_geometry_calculate_distances.BenchmarkTransportGeometryCalculateDistances.time_calculate_distance_line({'packet': {'nu_line': 0.2, 'is_last_line': False}, 'enable_full_relativity': True}) |
| | 7.68±0.01s | 7.68±0.01s | 1 | transport_montecarlo_interaction.BenchmarkMontecarloMontecarloNumbaInteraction.time_line_emission({'mu': 0.8599443103322428, 'emission_line_id': 1000, 'energy': 0.9114437898710559}) |
| | 7.73±0.03s | 7.73±0.03s | 1 | transport_montecarlo_interaction.BenchmarkMontecarloMontecarloNumbaInteraction.time_line_scatter(<LineInteractionType.MACROATOM: 2>) |
| | 7.68±0s | 7.68±0s | 1 | transport_montecarlo_interaction.BenchmarkMontecarloMontecarloNumbaInteraction.time_line_scatter(<LineInteractionType.SCATTER: 0>) |
| | 3.49±0.03s | 3.49±0.03s | 1 | transport_montecarlo_interaction.BenchmarkMontecarloMontecarloNumbaInteraction.time_thomson_scatter |
| | 4.35±0.03s | 4.35±0.03s | 1 | transport_montecarlo_main_loop.BenchmarkTransportMontecarloMainLoop.time_montecarlo_main_loop |
| | 2.14±0m | 2.14±0m | 1 | transport_montecarlo_numba_formal_integral_p.BenchmarkMontecarloMontecarloNumbaNumbaFormalIntegral.time_FormalIntegrator_functions |
| | 1.61±0.6μs | 1.61±0.6μs | 1 | transport_montecarlo_numba_formal_integral_p.BenchmarkMontecarloMontecarloNumbaNumbaFormalIntegral.time_calculate_p_values(100) |
| | 2.49±0.6μs | 2.49±0.6μs | 1 | transport_montecarlo_numba_formal_integral_p.BenchmarkMontecarloMontecarloNumbaNumbaFormalIntegral.time_calculate_p_values(1000) |
| | 9.83±0.6μs | 9.83±0.6μs | 1 | transport_montecarlo_numba_formal_integral_p.BenchmarkMontecarloMontecarloNumbaNumbaFormalIntegral.time_calculate_p_values(10000) |
| | 749±4ms | 749±4ms | 1 | transport_montecarlo_numba_formal_integral_p.BenchmarkMontecarloMontecarloNumbaNumbaFormalIntegral.time_calculate_z(0.0, array([0. , 0.5, 1. ])) |
| | 753±4ms | 753±4ms | 1 | transport_montecarlo_numba_formal_integral_p.BenchmarkMontecarloMontecarloNumbaNumbaFormalIntegral.time_calculate_z(0.0, array([1. , 1.5, 2. ])) |
| | 739±0.5ms | 739±0.5ms | 1 | transport_montecarlo_numba_formal_integral_p.BenchmarkMontecarloMontecarloNumbaNumbaFormalIntegral.time_calculate_z(0.5, array([0. , 0.5, 1. ])) |
| | 739±2ms | 739±2ms | 1 | transport_montecarlo_numba_formal_integral_p.BenchmarkMontecarloMontecarloNumbaNumbaFormalIntegral.time_calculate_z(0.5, array([1. , 1.5, 2. ])) |
| | 735±9ms | 735±9ms | 1 | transport_montecarlo_numba_formal_integral_p.BenchmarkMontecarloMontecarloNumbaNumbaFormalIntegral.time_calculate_z(1.0, array([0. , 0.5, 1. ])) |
| | 757±3ms | 757±3ms | 1 | transport_montecarlo_numba_formal_integral_p.BenchmarkMontecarloMontecarloNumbaNumbaFormalIntegral.time_calculate_z(1.0, array([1. , 1.5, 2. ])) |
| | 651±200ns | 651±200ns | 1 | transport_montecarlo_numba_formal_integral_p.BenchmarkMontecarloMontecarloNumbaNumbaFormalIntegral.time_intensity_black_body({'nu': 0, 'temperature': 1}) |
| | 691±200ns | 691±200ns | 1 | transport_montecarlo_numba_formal_integral_p.BenchmarkMontecarloMontecarloNumbaNumbaFormalIntegral.time_intensity_black_body({'nu': 1, 'temperature': 1}) |
| | 721±300ns | 721±300ns | 1 | transport_montecarlo_numba_formal_integral_p.BenchmarkMontecarloMontecarloNumbaNumbaFormalIntegral.time_intensity_black_body({'nu': 100000000000000.0, 'temperature': 10000.0}) |
| | 3.52±0.9μs | 3.52±0.9μs | 1 | transport_montecarlo_numba_formal_integral_p.BenchmarkMontecarloMontecarloNumbaNumbaFormalIntegral.time_trapezoid_integration(100.0) |
| | 9.16±1μs | 9.16±1μs | 1 | transport_montecarlo_numba_formal_integral_p.BenchmarkMontecarloMontecarloNumbaNumbaFormalIntegral.time_trapezoid_integration(1000.0) |
| | 64.9±1μs | 64.9±1μs | 1 | transport_montecarlo_numba_formal_integral_p.BenchmarkMontecarloMontecarloNumbaNumbaFormalIntegral.time_trapezoid_integration(10000.0) |
| | 651±10μs | 651±10μs | 1 | transport_montecarlo_numba_formal_integral_p.BenchmarkMontecarloMontecarloNumbaNumbaFormalIntegral.time_trapezoid_integration(100000.0) |
| | 1.46±0.01s | 1.46±0.01s | 1 | transport_montecarlo_numba_interface.BenchmarkMontecarloMontecarloNumbaNumbaInterface.time_opacity_state_initialize('macroatom') |
| | 1.46±0s | 1.46±0s | 1 | transport_montecarlo_numba_interface.BenchmarkMontecarloMontecarloNumbaNumbaInterface.time_opacity_state_initialize('scatter') |
| | 667±200ns | 667±200ns | 1 | transport_montecarlo_opacities.BenchmarkMontecarloMontecarloNumbaOpacities.time_compton_opacity_calculation(100000.0, 255.5) |
| | 621±200ns | 621±200ns | 1 | transport_montecarlo_opacities.BenchmarkMontecarloMontecarloNumbaOpacities.time_compton_opacity_calculation(100000.0, 511.0) |
| | 636±200ns | 636±200ns | 1 | transport_montecarlo_opacities.BenchmarkMontecarloMontecarloNumbaOpacities.time_compton_opacity_calculation(100000.0, 5110000000.0) |
| | 606±200ns | 606±200ns | 1 | transport_montecarlo_opacities.BenchmarkMontecarloMontecarloNumbaOpacities.time_compton_opacity_calculation(100000000000.0, 255.5) |
| | 626±200ns | 626±200ns | 1 | transport_montecarlo_opacities.BenchmarkMontecarloMontecarloNumbaOpacities.time_compton_opacity_calculation(100000000000.0, 511.0) |
| | 641±200ns | 641±200ns | 1 | transport_montecarlo_opacities.BenchmarkMontecarloMontecarloNumbaOpacities.time_compton_opacity_calculation(100000000000.0, 5110000000.0) |
| | 617±200ns | 617±200ns | 1 | transport_montecarlo_opacities.BenchmarkMontecarloMontecarloNumbaOpacities.time_compton_opacity_calculation(1000000000000000.0, 255.5) |
| | 631±200ns | 631±200ns | 1 | transport_montecarlo_opacities.BenchmarkMontecarloMontecarloNumbaOpacities.time_compton_opacity_calculation(1000000000000000.0, 511.0) |
| | 646±200ns | 646±200ns | 1 | transport_montecarlo_opacities.BenchmarkMontecarloMontecarloNumbaOpacities.time_compton_opacity_calculation(1000000000000000.0, 5110000000.0) |
| | 696±200ns | 696±200ns | 1 | transport_montecarlo_opacities.BenchmarkMontecarloMontecarloNumbaOpacities.time_pair_creation_opacity_calculation({'Ejecta_density': 0.01, 'Energy': 255.5, 'Iron_group_fraction': 0.0}) |
| | 706±300ns | 706±300ns | 1 | transport_montecarlo_opacities.BenchmarkMontecarloMontecarloNumbaOpacities.time_pair_creation_opacity_calculation({'Ejecta_density': 0.01, 'Energy': 255.5, 'Iron_group_fraction': 0.5}) |
| | 691±200ns | 691±200ns | 1 | transport_montecarlo_opacities.BenchmarkMontecarloMontecarloNumbaOpacities.time_pair_creation_opacity_calculation({'Ejecta_density': 0.01, 'Energy': 511.0, 'Iron_group_fraction': 0.5}) |
| | 722±200ns | 722±200ns | 1 | transport_montecarlo_opacities.BenchmarkMontecarloMontecarloNumbaOpacities.time_pair_creation_opacity_calculation({'Ejecta_density': 0.01, 'Energy': 5110000000.0, 'Iron_group_fraction': 0.0}) |
| | 702±200ns | 702±200ns | 1 | transport_montecarlo_opacities.BenchmarkMontecarloMontecarloNumbaOpacities.time_pair_creation_opacity_calculation({'Ejecta_density': 100000.0, 'Energy': 255.5, 'Iron_group_fraction': 1.0}) |
| | 681±200ns | 681±200ns | 1 | transport_montecarlo_opacities.BenchmarkMontecarloMontecarloNumbaOpacities.time_photoabsorption_opacity_calculation({'Ejecta_density': 0.01, 'Energy': 255.5, 'Iron_group_fraction': 0.0}) |
| | 741±200ns | 741±200ns | 1 | transport_montecarlo_opacities.BenchmarkMontecarloMontecarloNumbaOpacities.time_photoabsorption_opacity_calculation({'Ejecta_density': 0.01, 'Energy': 255.5, 'Iron_group_fraction': 0.5}) |
| | 726±200ns | 726±200ns | 1 | transport_montecarlo_opacities.BenchmarkMontecarloMontecarloNumbaOpacities.time_photoabsorption_opacity_calculation({'Ejecta_density': 0.01, 'Energy': 511.0, 'Iron_group_fraction': 0.5}) |
| | 751±200ns | 751±200ns | 1 | transport_montecarlo_opacities.BenchmarkMontecarloMontecarloNumbaOpacities.time_photoabsorption_opacity_calculation({'Ejecta_density': 0.01, 'Energy': 5110000000.0, 'Iron_group_fraction': 0.0}) |
| | 701±200ns | 701±200ns | 1 | transport_montecarlo_opacities.BenchmarkMontecarloMontecarloNumbaOpacities.time_photoabsorption_opacity_calculation({'Ejecta_density': 100000.0, 'Energy': 255.5, 'Iron_group_fraction': 1.0}) |
| | 711±200ns | 711±200ns | 1 | transport_montecarlo_packet.BenchmarkMontecarloMontecarloNumbaPacket.time_calculate_distance_electron({'electron_density': 1.0, 'tua_event': 10000000000.0}) |
| | 667±200ns | 667±200ns | 1 | transport_montecarlo_packet.BenchmarkMontecarloMontecarloNumbaPacket.time_calculate_distance_electron({'electron_density': 1e-05, 'tua_event': 10000000000.0}) |
| | 677±200ns | 677±200ns | 1 | transport_montecarlo_packet.BenchmarkMontecarloMontecarloNumbaPacket.time_calculate_tau_electron({'electron_density': -1, 'distance': 0}) |
| | 721±300ns | 721±300ns | 1 | transport_montecarlo_packet.BenchmarkMontecarloMontecarloNumbaPacket.time_calculate_tau_electron({'electron_density': -10000000000.0, 'distance': -10000000000.0}) |
| | 697±200ns | 697±200ns | 1 | transport_montecarlo_packet.BenchmarkMontecarloMontecarloNumbaPacket.time_calculate_tau_electron({'electron_density': 10000000000.0, 'distance': 10000000000.0}) |
| | 696±200ns | 696±200ns | 1 | transport_montecarlo_packet.BenchmarkMontecarloMontecarloNumbaPacket.time_calculate_tau_electron({'electron_density': 1e-05, 'distance': 1.0}) |
| | 360±100ns | 360±100ns | 1 | transport_montecarlo_packet.BenchmarkMontecarloMontecarloNumbaPacket.time_get_random_mu |
| | 2.15±0s | 2.15±0s | 1 | transport_montecarlo_packet.BenchmarkMontecarloMontecarloNumbaPacket.time_move_packet_across_shell_boundary_emitted({'current_shell_id': 132, 'delta_shell': 11, 'no_of_shells': 132}) |
| | 2.20±0.01s | 2.20±0.01s | 1 | transport_montecarlo_packet.BenchmarkMontecarloMontecarloNumbaPacket.time_move_packet_across_shell_boundary_emitted({'current_shell_id': 132, 'delta_shell': 2, 'no_of_shells': 133}) |
| | 2.17±0.04s | 2.17±0.04s | 1 | transport_montecarlo_packet.BenchmarkMontecarloMontecarloNumbaPacket.time_move_packet_across_shell_boundary_increment({'current_shell_id': 132, 'delta_shell': -1, 'no_of_shells': 199}) |
| | 2.19±0.04s | 2.19±0.04s | 1 | transport_montecarlo_packet.BenchmarkMontecarloMontecarloNumbaPacket.time_move_packet_across_shell_boundary_increment({'current_shell_id': 132, 'delta_shell': 0, 'no_of_shells': 132}) |
| | 2.20±0.02s | 2.20±0.02s | 1 | transport_montecarlo_packet.BenchmarkMontecarloMontecarloNumbaPacket.time_move_packet_across_shell_boundary_reabsorbed({'current_shell_id': 132, 'delta_shell': 132, 'no_of_shells': 132}) |
| | 2.15±0s | 2.15±0s | 1 | transport_montecarlo_packet.BenchmarkMontecarloMontecarloNumbaPacket.time_move_packet_across_shell_boundary_reabsorbed({'current_shell_id': 132, 'delta_shell': 133, 'no_of_shells': 133}) |
| | 37.6±5μs | 37.6±5μs | 1 | transport_montecarlo_packet.BenchmarkMontecarloMontecarloNumbaPacket.time_update_line_estimators({'cur_line_id': 0, 'distance_trace': 0, 'time_explosion': 52000000.0, 'enable_full_relativity': True}) |
| | 37.2±5μs | 37.2±5μs | 1 | transport_montecarlo_packet.BenchmarkMontecarloMontecarloNumbaPacket.time_update_line_estimators({'cur_line_id': 0, 'distance_trace': 1000000000000.0, 'time_explosion': 52000000.0, 'enable_full_relativity': True}) |
| | 38.4±5μs | 38.4±5μs | 1 | transport_montecarlo_packet.BenchmarkMontecarloMontecarloNumbaPacket.time_update_line_estimators({'cur_line_id': 1, 'distance_trace': 100000.0, 'time_explosion': 10000000000.0, 'enable_full_relativity': False}) |
| | 1.43±0.01s | 1.43±0.01s | 1 | transport_montecarlo_r_packet.BenchmarkMontecarloMontecarloNumbaRPacket.time_rpacket_trackers_to_dataframe |
| | 6.37±0.02s | 6.37±0.02s | 1 | transport_montecarlo_single_packet_loop.BenchmarkMontecarloMontecarloNumbaVpacket.time_single_packet_loop |
| | 5.02±0.02s | 5.02±0.02s | 1 | transport_montecarlo_vpacket.BenchmarkMontecarloMontecarloNumbaVpacket.time_trace_bad_vpacket |
| | 5.07±0.02s | 5.07±0.02s | 1 | transport_montecarlo_vpacket.BenchmarkMontecarloMontecarloNumbaVpacket.time_trace_vpacket |
| | 4.96±0.03s | 4.96±0.03s | 1 | transport_montecarlo_vpacket.BenchmarkMontecarloMontecarloNumbaVpacket.time_trace_vpacket_volley({'tau_russian': 10.0, 'survival_possibility': 0.0}) |
| | 5.01±0.01s | 5.01±0.01s | 1 | transport_montecarlo_vpacket.BenchmarkMontecarloMontecarloNumbaVpacket.time_trace_vpacket_volley({'tau_russian': 15.0, 'survival_possibility': 0.1}) |
| | 3.74±0.02s | 3.74±0.02s | 1 | transport_montecarlo_vpacket.BenchmarkMontecarloMontecarloNumbaVpacket.time_trace_vpacket_within_shell |
If you want to see the graph of the results, you can check it here |
The benchmark run is successful in these changes. |
I notice the before and after commits are still the same in the bot comment. Should that have been resolved here? |
Signed-off-by: Asish Kumar <[email protected]>
Signed-off-by: Asish Kumar <[email protected]>
Signed-off-by: Asish Kumar <[email protected]>
Signed-off-by: Asish Kumar <[email protected]>
Signed-off-by: Asish Kumar <[email protected]>
new action run on the existing benchmarks.yml -> https://github.com/officialasishkumar/tardis/actions/runs/10077232306/job/27859281972 |
I have ironed out all the bugs. So this can be merged to get the benchmarks working. cc: @andrewfullard |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good job fixing the benchmarks, please remove the test items
@@ -61,6 +62,9 @@ def single_packet_loop( | |||
This function does not return anything but changes the r_packet object | |||
and if virtual packets are requested - also updates the vpacket_collection | |||
""" | |||
|
|||
for i in range(200): | |||
print("hello") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please remove this
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
sorry, my bad. I will fix it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
removed
@@ -30,6 +30,7 @@ | |||
) | |||
from tardis.transport.montecarlo.vpacket import trace_vpacket_volley | |||
|
|||
import time |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please remove this
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
removed
Signed-off-by: Asish Kumar <[email protected]>
fcf70f5
to
4d73d7c
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me too!
📝 Description
Type: 🪲
bugfix
📌 Resources
Examples, notebooks, and links to useful references.
🚦 Testing
How did you test these changes?
☑️ Checklist
build_docs
label