Migrate to Regression Data Completely #2808

atharva-2001 · 2024-08-23T05:02:14Z

📝 Description

Type: ☣️ breaking change | 🚦 testing | 🎢 infrastructure

This PR aims to remove references of refdata from the tardis repo.
tardis-sn/tardis-regression-data#21 and tardis-sn/tardis-regression-data#22 will need to be merged before in order for tests to pass.

📌 Resources

Examples, notebooks, and links to useful references.

🚦 Testing

How did you test these changes?

Testing pipeline
Other method (describe)
My changes can't be tested (explain why)

☑️ Checklist

I requested two reviewers for this pull request
I updated the documentation according to my changes
I built the documentation by applying the build_docs label

Note: If you are not allowed to perform any of these actions, ping (@) a contributor.

tardis-bot · 2024-08-23T05:03:49Z

*beep* *bop*
Hi human,
I ran ruff on the latest commit (8b6fda3).
Here are the outputs produced.
Results can also be downloaded as artifacts here.
Summarised output:

23	E999  	[ ] SyntaxError: Expected an expression
6	UP015 	[*] Unnecessary open mode parameters
5	W291  	[*] Trailing whitespace
4	I001  	[*] Import block is un-sorted or un-formatted
4	F401  	[*] `astropy.constants` imported but unused
2	FLY002	[*] Consider `f"{hash_string}_{nlte_prop}"` instead of string join
1	B018  	[ ] Found useless expression. Either assign it to a variable or remove it.
1	G004  	[ ] Logging statement uses f-string
1	INP001	[ ] File `tardis/io/model/readers/tests/test_arepo_parser.py` is part of an implicit namespace package. Add an `__init__.py`.
1	W292  	[*] No newline at end of file
1	W605  	[*] Invalid escape sequence: `\g`
1	F403  	[ ] `from tardis.tests.fixtures.atom_data import *` used; unable to detect undefined names
1	F821  	[ ] Undefined name `pytest_report_header`
1	PGH004	[ ] Use specific rule codes when using `noqa`

Complete output(might be large):

.github/actions/setup_lfs/action.yml:5:8: E999 SyntaxError: Expected an expression
.github/workflows/archive/dispatcher.yml:2:4: E999 SyntaxError: Expected an expression
.github/workflows/build-docs.yml:7:4: E999 SyntaxError: Expected an expression
.github/workflows/pre-release.yml:7:4: E999 SyntaxError: Expected an expression
.github/workflows/tests.yml:7:4: E999 SyntaxError: Expected an expression
asv.conf.json:1:1: B018 Found useless expression. Either assign it to a variable or remove it.
asv.conf.json:32:2: W292 [*] No newline at end of file
docs/io/configuration/components/models/data/config_init_trad.yml:2:26: E999 SyntaxError: Simple statements must be separated by newlines or semicolons
docs/io/configuration/components/models/data/config_no_init_trad.yml:2:26: E999 SyntaxError: Simple statements must be separated by newlines or semicolons
docs/resources/code_comparison/plasma_compare/tardis_example.yml:2:26: E999 SyntaxError: Simple statements must be separated by newlines or semicolons
tardis/conftest.py:12:1: F403 `from tardis.tests.fixtures.atom_data import *` used; unable to detect undefined names
tardis/conftest.py:30:49: PGH004 Use specific rule codes when using `noqa`
tardis/conftest.py:32:9: F821 Undefined name `pytest_report_header`
tardis/energy_input/docs/working_gamma_ray_test.ipynb:cell 1:2:1: I001 [*] Import block is un-sorted or un-formatted
tardis/energy_input/docs/working_gamma_ray_test.ipynb:cell 1:3:18: F401 [*] `pandas` imported but unused
tardis/energy_input/docs/working_gamma_ray_test.ipynb:cell 1:6:29: F401 [*] `astropy.constants` imported but unused
tardis/energy_input/docs/working_gamma_ray_test.ipynb:cell 1:7:25: F401 [*] `astropy.units` imported but unused
tardis/energy_input/docs/working_gamma_ray_test.ipynb:cell 2:2:1: I001 [*] Import block is un-sorted or un-formatted
tardis/energy_input/docs/working_gamma_ray_test.ipynb:cell 7:5:70: W291 [*] Trailing whitespace
tardis/energy_input/docs/working_gamma_ray_test.ipynb:cell 9:3:96: W291 [*] Trailing whitespace
tardis/energy_input/docs/working_gamma_ray_test.ipynb:cell 9:4:99: W291 [*] Trailing whitespace
tardis/energy_input/docs/working_gamma_ray_test.ipynb:cell 9:6:78: W291 [*] Trailing whitespace
tardis/energy_input/docs/working_gamma_ray_test.ipynb:cell 10:2:65: W605 [*] Invalid escape sequence: `\g`
tardis/io/model/parse_atom_data.py:40:21: G004 Logging statement uses f-string
tardis/io/model/readers/tests/data/csvy_test_config.yml:2:26: E999 SyntaxError: Simple statements must be separated by newlines or semicolons
tardis/io/model/readers/tests/test_arepo_parser.py:1:1: INP001 File `tardis/io/model/readers/tests/test_arepo_parser.py` is part of an implicit namespace package. Add an `__init__.py`.
tardis/io/model/readers/tests/test_arepo_parser.py:1:1: I001 [*] Import block is un-sorted or un-formatted
tardis/io/model/readers/tests/test_arepo_parser.py:16:10: UP015 [*] Unnecessary open mode parameters
tardis/io/model/readers/tests/test_arepo_parser.py:48:10: UP015 [*] Unnecessary open mode parameters
tardis/io/model/readers/tests/test_arepo_parser.py:59:10: UP015 [*] Unnecessary open mode parameters
tardis/io/model/readers/tests/test_arepo_parser.py:89:10: UP015 [*] Unnecessary open mode parameters
tardis/io/model/readers/tests/test_arepo_parser.py:99:10: UP015 [*] Unnecessary open mode parameters
tardis/io/model/readers/tests/test_arepo_parser.py:109:10: UP015 [*] Unnecessary open mode parameters
tardis/model/tests/data/branch85_csvy.yml:2:26: E999 SyntaxError: Simple statements must be separated by newlines or semicolons
tardis/model/tests/data/branch85_old_config.yml:2:26: E999 SyntaxError: Simple statements must be separated by newlines or semicolons
tardis/model/tests/data/csvy_model_to_test_abundances.yml:2:26: E999 SyntaxError: Simple statements must be separated by newlines or semicolons
tardis/model/tests/data/exponential_csvy.yml:2:26: E999 SyntaxError: Simple statements must be separated by newlines or semicolons
tardis/model/tests/data/exponential_old_config.yml:2:26: E999 SyntaxError: Simple statements must be separated by newlines or semicolons
tardis/model/tests/data/model_full_csvy.yml:2:26: E999 SyntaxError: Simple statements must be separated by newlines or semicolons
tardis/model/tests/data/model_full_old_config.yml:2:26: E999 SyntaxError: Simple statements must be separated by newlines or semicolons
tardis/model/tests/data/powerlaw_csvy.yml:2:26: E999 SyntaxError: Simple statements must be separated by newlines or semicolons
tardis/model/tests/data/powerlaw_old_config.yml:2:26: E999 SyntaxError: Simple statements must be separated by newlines or semicolons
tardis/model/tests/data/radiative_csvy.yml:2:26: E999 SyntaxError: Simple statements must be separated by newlines or semicolons
tardis/model/tests/data/radiative_old_config.yml:2:26: E999 SyntaxError: Simple statements must be separated by newlines or semicolons
tardis/model/tests/data/uniform_csvy.yml:1:69: W291 [*] Trailing whitespace
tardis/model/tests/data/uniform_csvy.yml:2:26: E999 SyntaxError: Simple statements must be separated by newlines or semicolons
tardis/model/tests/data/uniform_old_config.yml:2:26: E999 SyntaxError: Simple statements must be separated by newlines or semicolons
tardis/plasma/tests/data/config_init_trad.yml:2:26: E999 SyntaxError: Simple statements must be separated by newlines or semicolons
tardis/plasma/tests/test_complete_plasmas.py:149:27: FLY002 Consider `f"{hash_string}_{prop}"` instead of string join
tardis/plasma/tests/test_complete_plasmas.py:154:39: FLY002 Consider `f"{hash_string}_{nlte_prop}"` instead of string join
tardis/tests/fixtures/atom_data.py:1:1: I001 [*] Import block is un-sorted or un-formatted
tardis/tests/fixtures/atom_data.py:2:21: F401 [*] `pathlib.Path` imported but unused
Found 52 errors.
[*] 21 fixable with the `--fix` option (2 hidden fixes can be enabled with the `--unsafe-fixes` option).

review-notebook-app · 2024-08-23T07:46:12Z

Check out this pull request on

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

tardis-bot · 2024-08-23T08:23:21Z

*beep* *bop*

Hi, human.

The docs workflow has failed ❌

Click here to see the build log.

…re release, correct build docs workflow

docs/io/configuration/components/models/data/config_init_trad.yml

andrewfullard

I'm happy with this, I just want @wkerzendorf to sign off as well on the regression data PRs, then everything here should work as expected.

andrewfullard · 2024-09-03T14:24:04Z

Can you fix the benchmarks? Docs as well.

tardis-bot · 2024-09-04T13:34:19Z

*beep* *bop*
Hi human,
I ran benchmarks as you asked comparing master (260207c) and the latest commit (8b6fda3).
Here are the logs produced by ASV.
Results can also be downloaded as artifacts here.

Significantly changed benchmarks:

All benchmarks:

Benchmarks that have stayed the same:

| Change   | Before [260207cb] <master>   | After [8b6fda30]    | Ratio   | Benchmark (Parameter)                                                                                                               |
|----------|------------------------------|---------------------|---------|-------------------------------------------------------------------------------------------------------------------------------------|
|          | 2.03±1μs                     | 2.50±1μs            | ~1.23   | transport_montecarlo_estimators_radfield_estimator_calcs.BenchmarkMontecarloMontecarloNumbaPacket.time_update_line_estimators       |
|          | 3.10±0.5μs                   | 3.80±0.6μs          | ~1.23   | transport_montecarlo_vpacket.BenchmarkMontecarloMontecarloNumbaVpacket.time_trace_bad_vpacket                                       |
|          | 44.2±20μs                    | 51.8±30μs           | ~1.17   | transport_montecarlo_interaction.BenchmarkTransportMontecarloInteraction.time_line_scatter                                          |
|          | 1.73±0.02ms                  | 1.95±0.01ms         | ~1.13   | transport_montecarlo_main_loop.BenchmarkTransportMontecarloMontecarloMainLoop.time_montecarlo_main_loop                             |
|          | 491±200ns                    | 551±100ns           | ~1.12   | opacities_opacity.BenchmarkMontecarloMontecarloNumbaOpacities.time_compton_opacity_calculation                                      |
|          | 1.58±0.3μs                   | 1.38±0.4μs          | ~0.87   | transport_geometry_calculate_distances.BenchmarkTransportGeometryCalculateDistances.time_calculate_distance_line                    |
|          | 3.46±0.01ms                  | 2.77±0.03ms         | ~0.80   | opacities_opacity_state.BenchmarkOpacitiesOpacityState.time_opacity_state_initialize('scatter')                                     |
|          | 11.0±3μs                     | 7.30±2μs            | ~0.66   | transport_montecarlo_vpacket.BenchmarkMontecarloMontecarloNumbaVpacket.time_trace_vpacket_volley                                    |
|          | 6.50±0.8μs                   | 6.87±0.6μs          | 1.06    | transport_montecarlo_vpacket.BenchmarkMontecarloMontecarloNumbaVpacket.time_trace_vpacket                                           |
|          | 23.2±6μs                     | 23.9±6μs            | 1.03    | transport_montecarlo_packet_trackers.BenchmarkTransportMontecarloPacketTrackers.time_generate_rpacket_last_interaction_tracker_list |
|          | 2.77±0.4ms                   | 2.81±0.5ms          | 1.02    | transport_montecarlo_single_packet_loop.BenchmarkTransportMontecarloSinglePacketLoop.time_single_packet_loop                        |
|          | 2.07±0m                      | 2.10±0.01m          | 1.01    | spectrum_formal_integral.BenchmarkTransportMontecarloFormalIntegral.time_FormalIntegrator_functions                                 |
|          | 204±0.2ns                    | 206±0.6ns           | 1.01    | spectrum_formal_integral.BenchmarkTransportMontecarloFormalIntegral.time_intensity_black_body                                       |
|          | 1.17±0μs                     | 1.18±0μs            | 1.01    | transport_geometry_calculate_distances.BenchmarkTransportGeometryCalculateDistances.time_calculate_distance_boundary                |
|          | 63.2±0.2ms                   | 64.1±0.06ms         | 1.01    | transport_montecarlo_packet_trackers.BenchmarkTransportMontecarloPacketTrackers.time_rpacket_trackers_to_dataframe                  |
|          | 601±100ns                    | 601±100ns           | 1.00    | opacities_opacity.BenchmarkMontecarloMontecarloNumbaOpacities.time_pair_creation_opacity_calculation                                |
|          | 30.6±0.01μs                  | 30.6±0.06μs         | 1.00    | transport_montecarlo_packet_trackers.BenchmarkTransportMontecarloPacketTrackers.time_generate_rpacket_tracker_list                  |
|          | 3.26±0.5μs                   | 3.25±0.4μs          | 1.00    | transport_montecarlo_vpacket.BenchmarkMontecarloMontecarloNumbaVpacket.time_trace_vpacket_within_shell                              |
|          | 3.92±0.03ms                  | 3.79±0.02ms         | 0.97    | opacities_opacity_state.BenchmarkOpacitiesOpacityState.time_opacity_state_initialize('macroatom')                                   |
|          | 561±100ns                    | 541±200ns           | 0.96    | opacities_opacity.BenchmarkMontecarloMontecarloNumbaOpacities.time_photoabsorption_opacity_calculation                              |
|          | 40.8±0.05s                   | 39.2±0.08s          | 0.96    | run_tardis.BenchmarkRunTardis.time_run_tardis                                                                                       |
|          | 1.09±0m                      | 1.05±0m             | 0.96    | run_tardis.BenchmarkRunTardis.time_run_tardis_rpacket_tracking                                                                      |
|          | 44.8±20μs                    | 42.1±20μs           | 0.94    | transport_montecarlo_interaction.BenchmarkTransportMontecarloInteraction.time_line_emission                                         |
|          | 742±0.5ns                    | 697±1ns             | 0.94    | transport_montecarlo_interaction.BenchmarkTransportMontecarloInteraction.time_thomson_scatter                                       |

If you want to see the graph of the results, you can check it here

codecov · 2024-09-06T06:22:52Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 70.79%. Comparing base (be4ec9a) to head (8b6fda3).
Report is 3 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2808      +/-   ##
==========================================
- Coverage   70.96%   70.79%   -0.18%     
==========================================
  Files         209      209              
  Lines       15638    15637       -1     
==========================================
- Hits        11098    11070      -28     
- Misses       4540     4567      +27

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

andrewfullard · 2024-09-06T14:15:58Z

Do you know why docs are failing?

atharva-2001 · 2024-09-06T16:51:47Z

I think it is because of the YAML file not being updated because it is a pull request target event. The YAML file was picked up from the master branch instead of this branch.

…

On Fri, 6 Sep 2024 at 7:46 PM, Andrew ***@***.***> wrote: Do you know why docs are failing? — Reply to this email directly, view it on GitHub <#2808 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ANKOCXAKNDMI54URPKGDUQDZVG2LJAVCNFSM6AAAAABM7N3M7GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMZUGE3DEMBWGY> . You are receiving this because you authored the thread.Message ID: ***@***.***>

atharva-2001 · 2024-09-06T16:52:09Z

I think it should all work once the PR is merged! On Fri, 6 Sep 2024 at 10:21 PM, Atharva Arya ***@***.***> wrote:

…

I think it is because of the YAML file not being updated because it is a pull request target event. The YAML file was picked up from the master branch instead of this branch. On Fri, 6 Sep 2024 at 7:46 PM, Andrew ***@***.***> wrote: > Do you know why docs are failing? > > — > Reply to this email directly, view it on GitHub > <#2808 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/ANKOCXAKNDMI54URPKGDUQDZVG2LJAVCNFSM6AAAAABM7N3M7GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMZUGE3DEMBWGY> > . > You are receiving this because you authored the thread.Message ID: > ***@***.***> >

Remove refdata reference from .py files, tests pass

9fa3464

atharva-2001 added 2 commits August 23, 2024 13:06

Error message fix

15c4ede

Minor correction in config file comments

dd9ae5a

Remove refdata reference from tests

825233e

atharva-2001 mentioned this pull request Aug 23, 2024

Migrate to Regression Data Completely tardis-sn/tardis-regression-data#21

Merged

6 tasks

atharva-2001 marked this pull request as ready for review August 23, 2024 08:20

atharva-2001 added 3 commits August 23, 2024 14:17

Archive dispatcher for compare refdata, remove compare refdata from p…

0a61b6d

…re release, correct build docs workflow

Apply ruff on workflow files

b026803

Apply black and ruff in python files

bad18e3

andrewfullard requested review from andrewfullard and wkerzendorf August 23, 2024 13:28

andrewfullard reviewed Aug 23, 2024

View reviewed changes

docs/io/configuration/components/models/data/config_init_trad.yml Outdated Show resolved Hide resolved

Use full name of the repository

214fdeb

atharva-2001 requested a review from andrewfullard August 23, 2024 13:55

Turn spectrum tests into a module

2706faa

atharva-2001 requested a review from jvshields September 2, 2024 06:26

andrewfullard reviewed Sep 3, 2024

View reviewed changes

atharva-2001 force-pushed the migrate-to-regression-1 branch from de61a7a to 32c57e1 Compare September 4, 2024 12:32

atharva-2001 added the build-docs label Sep 6, 2024

atharva-2001 added 2 commits September 6, 2024 12:14

Use regression data in asv config

3b9ed17

Upload hidden files

ec55ece

atharva-2001 force-pushed the migrate-to-regression-1 branch from b7256dc to ec55ece Compare September 6, 2024 06:46

atharva-2001 requested a review from andrewfullard September 6, 2024 13:58

atharva-2001 mentioned this pull request Sep 6, 2024

Remove rpacket_tracking flag from tests workflow #2746

Merged

6 tasks

andrewfullard previously approved these changes Sep 6, 2024

View reviewed changes

Merge branch 'master' into migrate-to-regression-1

8b6fda3

andrewfullard dismissed their stale review via 8b6fda3 September 9, 2024 13:59

andrewfullard self-requested a review September 9, 2024 14:26

andrewfullard approved these changes Sep 9, 2024

View reviewed changes

This was referenced Sep 9, 2024

Update line_interaction_in_r after moving the packet #2811

Merged

Handling for qgridnext installation #2819

Merged

wkerzendorf merged commit 31d352a into tardis-sn:master Sep 11, 2024
16 of 18 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Migrate to Regression Data Completely #2808

Migrate to Regression Data Completely #2808

atharva-2001 commented Aug 23, 2024 •

edited

Loading

tardis-bot commented Aug 23, 2024 •

edited

Loading

review-notebook-app bot commented Aug 23, 2024

tardis-bot commented Aug 23, 2024 •

edited

Loading

andrewfullard left a comment

andrewfullard commented Sep 3, 2024 •

edited

Loading

tardis-bot commented Sep 4, 2024 •

edited

Loading

codecov bot commented Sep 6, 2024 •

edited

Loading

andrewfullard commented Sep 6, 2024

atharva-2001 commented Sep 6, 2024 via email

atharva-2001 commented Sep 6, 2024 via email

Migrate to Regression Data Completely #2808

Migrate to Regression Data Completely #2808

Conversation

atharva-2001 commented Aug 23, 2024 • edited Loading

📝 Description

📌 Resources

🚦 Testing

☑️ Checklist

tardis-bot commented Aug 23, 2024 • edited Loading

review-notebook-app bot commented Aug 23, 2024

tardis-bot commented Aug 23, 2024 • edited Loading

andrewfullard left a comment

Choose a reason for hiding this comment

andrewfullard commented Sep 3, 2024 • edited Loading

tardis-bot commented Sep 4, 2024 • edited Loading

codecov bot commented Sep 6, 2024 • edited Loading

Codecov Report

andrewfullard commented Sep 6, 2024

atharva-2001 commented Sep 6, 2024 via email

atharva-2001 commented Sep 6, 2024 via email

atharva-2001 commented Aug 23, 2024 •

edited

Loading

tardis-bot commented Aug 23, 2024 •

edited

Loading

tardis-bot commented Aug 23, 2024 •

edited

Loading

andrewfullard commented Sep 3, 2024 •

edited

Loading

tardis-bot commented Sep 4, 2024 •

edited

Loading

codecov bot commented Sep 6, 2024 •

edited

Loading