[ENH] Access output in timeseries by its time or redshift #4717

cphyc · 2023-10-27T15:27:21Z

PR Summary

This provides the ability to quickly find the dataset that has the closest value to a given one (e.g. time or redshift). Example

import yt

# Load all RAMSES dataset in a folder - we assume here we have a cosmological simulation
ts = yt.load("/path/to/simulation/folder/output*")

# Get the snapshot that's as close as possible from z=2
ds = ts.get_by_redshift(2)

# Get the snapshot that's as close as possible from t=10Gyr after Big-Bang,
t = ts[0].quan(10, "Gyr")
ds = ts.get_by_time(t)

PR Checklist

New features are documented, with docstrings and narrative docs
Adds a test for any bugs fixed. Adds tests for new features.

neutrinoceros · 2023-10-27T15:46:27Z

Quick input:

I think this is a great idea
I'd prefer we avoid making it overly flexible: supporting the ability to search via arbitrary attributes may create unforeseen edge cases which will be impossible to test systematically. I think supporting search by time or redshift is sufficient for now, especially if it can be done in a way that doesn't involve typing the exact attribute names (IMO current_time is somewhat badly named, because its meaning seems excessively context-dependent)

matthewturk · 2023-11-01T17:03:50Z

Not sure I completely agree with @neutrinoceros about limiting the flexibility. The question the other day about flash datasets with particles would suggest there may be times when we don't know the attribute someone will be looking for.

Can you make it use "Price Is Right" rules? i.e., closest-without-going-over?

cphyc · 2023-11-01T17:16:56Z

I am not sure I understand what you mean by "Price Is Right" rule. Anyways, in its current state, it is already possible to query a dataset by attribute, but the method is hidden (_get_by_attribute), because “with great power comes great responsibility”.

Let me know what you think! I can always strip the leading _ to make it more user-facing, with an obvious comment in the docstring warning against misuses?

matthewturk · 2023-11-01T17:22:50Z

I mean, if I want the closet to z=1 but definitely on one side or the other, for instance. So having 0.99 take precedence over 1.01. (or even 0.95 over 1.001)

…

On Wed, Nov 1, 2023, 12:17 PM Corentin Cadiou ***@***.***> wrote: I am not sure I understand what you mean by "Price Is Right" rule. Anyways, in its current state, it is already possible to query a dataset by attribute, but the method is hidden (_get_by_attribute), because “with great power comes great responsibility”. Let me know what you think! I can always strip the leading _ to make it more user-facing, with an obvious comment in the docstring warning against misuses? — Reply to this email directly, view it on GitHub <#4717 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAVXOYJQUYAJMHRHRFESSDYCJ7ZHAVCNFSM6AAAAAA6S75QZ2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTOOBZGM2DKNZZGU> . You are receiving this because you commented.Message ID: ***@***.***>

cphyc · 2023-11-01T17:38:59Z

Oh I see, yes, I'll implement a side argument. To either get the output on the left or on the right!

neutrinoceros · 2023-11-01T17:45:06Z

I have nothing against that "the price is right rule", but I'd like to point out that left-to-right reading order isn't culturally neutral. I'd prefer that the interface be more inclusive.

cphyc · 2023-11-01T17:57:09Z

@neutrinoceros I've implemented a "smaller", "larger" or "nearest" switch so that there is no ambiguity arising from left-to-right vs. right-to-left reading order. Thanks for pointing it out!

matthewturk · 2023-11-01T18:49:25Z

I think the new language is better. That being said, it's not necessarily true that RTL languages have different number line concepts:

https://math.stackexchange.com/questions/2618537/do-right-to-left-readers-also-reverse-mathematical-concepts-like-number-line

There's also some evidence that filmmaking conventions (such as progression from the left side of the screen to the right) are also commonly used outside of left-to-right languages.

I will say that with redshift the entire situation is odd because the numeric progression is reversed from temporal progression. And I did choose redshift as my example... So I guess this is on me!

cphyc · 2023-11-02T09:13:19Z

@yt-fido test this please

neutrinoceros

The functionality is useful and the implementation is sound ! I have a couple questions and suggestions (mostly about type annotations)

neutrinoceros · 2023-11-25T10:14:07Z

doc/source/analyzing/time_series_analysis.rst

+   # This will fail if no output is found within 100 Myr
+   ds = ts.get_by_time((3, "Gyr"), tolerance=(100, "Myr"))
+   # Get the output at the time right before and after 3 Gyr
+   ds_before = ts.get_by_time((3, "Gyr"), side="smaller")


I still think the name of the argument could be clearer than side. What about prefer="smaller" ?

yt/data_objects/time_series.py

cphyc · 2023-11-27T16:37:11Z

@yt-fido test this please

cphyc · 2023-11-27T16:44:05Z

@yt-fido test this please

chrishavlin

Looks great!

I think the case of choosing a value exactly between two datasets needs to either explicitly handled or have its behavior defined in the docstring (see comments), but otherwise this is very useful!

yt/data_objects/tests/test_time_series.py

yt/data_objects/time_series.py

neutrinoceros · 2024-04-13T06:11:32Z

pre-commit.ci autofix

cphyc · 2024-04-16T08:12:17Z

@yt-fido test this please

neutrinoceros · 2024-04-16T09:54:11Z

@yt-fido Test this please

neutrinoceros · 2024-04-16T09:58:28Z

@yt-fido Test this please

neutrinoceros · 2024-04-16T11:51:03Z

@yt-fido Test this please

chrishavlin · 2024-10-02T16:50:56Z

Hey @cphyc , you still hoping for this to go into 4.4?

I'd still like some clarification on expected behavior when you choose a time exactly between two points while setting the prefer keyword (see #4717 (comment) ), but otherwise I think everything has been addressed?

… value

Co-authored-by: Clément Robert <[email protected]>

for more information, see https://pre-commit.ci

Co-authored-by: Chris Havlin <[email protected]>

cphyc · 2024-10-03T13:18:04Z

@chrishavlin should be good now :) I am now testing the behaviour when exactly halfway which is working as expected as far as I could tell.

chrishavlin · 2024-10-03T15:37:36Z

great! thanks, @cphyc !

chrishavlin · 2024-10-03T15:40:17Z

@neutrinoceros I think all your comments have been addressed, mind giving this a quick re-review when you have a moment?

neutrinoceros

Thanks for the ping ! Indeed my comments were all addressed !

neutrinoceros · 2024-10-03T16:12:31Z

Last thing before we merge this: @cphyc, can you re-title the PR to something a bit more descriptive that I can include directly in release notes ?

cphyc · 2024-10-03T16:50:53Z

Done!

cphyc added the new feature Something fun and new! label Oct 27, 2023

cphyc force-pushed the find-by-key branch from 10e57ba to e0e7136 Compare November 1, 2023 17:55

cphyc force-pushed the find-by-key branch from e0e7136 to a41c86a Compare November 1, 2023 17:58

cphyc marked this pull request as ready for review November 2, 2023 09:31

cphyc force-pushed the find-by-key branch from bb6a687 to 8072744 Compare November 3, 2023 15:30

neutrinoceros reviewed Nov 25, 2023

View reviewed changes

cphyc force-pushed the find-by-key branch from 57f7b5f to e53668d Compare November 27, 2023 16:00

chrishavlin reviewed Nov 28, 2023

View reviewed changes

yt/data_objects/tests/test_time_series.py Show resolved Hide resolved

yt/data_objects/tests/test_time_series.py Show resolved Hide resolved

yt/data_objects/tests/test_time_series.py Show resolved Hide resolved

yt/data_objects/time_series.py Show resolved Hide resolved

cphyc added this to the 4.4.0 milestone Apr 12, 2024

cphyc force-pushed the find-by-key branch from 6c24010 to b45d47d Compare April 12, 2024 12:55

cphyc added 3 commits October 3, 2024 15:04

Add functionality to find a dataset by its closest value

e70a8aa

Test the ability to get output from key

a967fce

Only 'get_by_time' and '_by_redshift' are user-facing

42fd1ae

cphyc and others added 16 commits October 3, 2024 15:04

Fix typos in docstring

49c6910

Match by filename

4a83b76

Short-circuit iterations for out-of-bounds and exact matches

4883f57

Make sure we can pass tuples for unitfull quantities

87a2686

Skip time_series for nose testing

559cab0

Provide 'side' to pick whether we want the closest, smaller or larger…

77e705b

… value

Add documentation

ad5a6c9

Update yt/data_objects/tests/test_time_series.py

652b685

Co-authored-by: Clément Robert <[email protected]>

Replace numpy.assert_raises with pytest's version

e5e609a

Import TYPE_CHECKING from typing

bc101e3

Remove outdated comment

cd8970e

Rename iH(igh) to the more canonical iR(ight)

7fba4c8

Rename side to more explicit 'prefer' kwa

8f05feb

Fix typing

199cfd7

Do not suggest very vague fix

385aeb2

[pre-commit.ci] auto fixes from pre-commit.com hooks

70a5714

for more information, see https://pre-commit.ci

cphyc force-pushed the find-by-key branch from ef91481 to b080a59 Compare October 3, 2024 13:05

Add extra test to check halfpoint

fe4642d

Co-authored-by: Chris Havlin <[email protected]>

cphyc force-pushed the find-by-key branch from b080a59 to fe4642d Compare October 3, 2024 13:16

chrishavlin approved these changes Oct 3, 2024

View reviewed changes

neutrinoceros approved these changes Oct 3, 2024

View reviewed changes

cphyc changed the title ~~Find output by its attribute~~ [ENH] Access output in timeseries by its time or redshift Oct 3, 2024

chrishavlin merged commit fcec2c5 into yt-project:main Oct 3, 2024
14 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ENH] Access output in timeseries by its time or redshift #4717

[ENH] Access output in timeseries by its time or redshift #4717

cphyc commented Oct 27, 2023 •

edited

Loading

neutrinoceros commented Oct 27, 2023 •

edited

Loading

matthewturk commented Nov 1, 2023

cphyc commented Nov 1, 2023

matthewturk commented Nov 1, 2023 via email

cphyc commented Nov 1, 2023

neutrinoceros commented Nov 1, 2023

cphyc commented Nov 1, 2023

matthewturk commented Nov 1, 2023

cphyc commented Nov 2, 2023

neutrinoceros left a comment

neutrinoceros Nov 25, 2023

cphyc commented Nov 27, 2023

cphyc commented Nov 27, 2023

chrishavlin left a comment

neutrinoceros commented Apr 13, 2024

cphyc commented Apr 16, 2024

neutrinoceros commented Apr 16, 2024

neutrinoceros commented Apr 16, 2024

neutrinoceros commented Apr 16, 2024

chrishavlin commented Oct 2, 2024

cphyc commented Oct 3, 2024

chrishavlin commented Oct 3, 2024

chrishavlin commented Oct 3, 2024

neutrinoceros left a comment

neutrinoceros commented Oct 3, 2024

cphyc commented Oct 3, 2024

[ENH] Access output in timeseries by its time or redshift #4717

[ENH] Access output in timeseries by its time or redshift #4717

Conversation

cphyc commented Oct 27, 2023 • edited Loading

PR Summary

PR Checklist

neutrinoceros commented Oct 27, 2023 • edited Loading

matthewturk commented Nov 1, 2023

cphyc commented Nov 1, 2023

matthewturk commented Nov 1, 2023 via email

cphyc commented Nov 1, 2023

neutrinoceros commented Nov 1, 2023

cphyc commented Nov 1, 2023

matthewturk commented Nov 1, 2023

cphyc commented Nov 2, 2023

neutrinoceros left a comment

Choose a reason for hiding this comment

neutrinoceros Nov 25, 2023

Choose a reason for hiding this comment

cphyc commented Nov 27, 2023

cphyc commented Nov 27, 2023

chrishavlin left a comment

Choose a reason for hiding this comment

neutrinoceros commented Apr 13, 2024

cphyc commented Apr 16, 2024

neutrinoceros commented Apr 16, 2024

neutrinoceros commented Apr 16, 2024

neutrinoceros commented Apr 16, 2024

chrishavlin commented Oct 2, 2024

cphyc commented Oct 3, 2024

chrishavlin commented Oct 3, 2024

chrishavlin commented Oct 3, 2024

neutrinoceros left a comment

Choose a reason for hiding this comment

neutrinoceros commented Oct 3, 2024

cphyc commented Oct 3, 2024

cphyc commented Oct 27, 2023 •

edited

Loading

neutrinoceros commented Oct 27, 2023 •

edited

Loading