revive out of order commit

[rohany]

There are cases (like the convergence check in Pennant for example), where in-order commit leads to false dependencies in an application. Reviving out-of-order commit for restricted cases (like during trace replays) would be beneficial when there isn't an easy workaround from the application.

[elliottslaughter]

As an example of the type of thing you have to do to work around this, see: https://gitlab.com/StanfordLegion/legion/-/merge_requests/1697

I'm not sure it's only about tracing either. Pretty much any time an application blocks on a future, this causes that future to carry a spurious dependency on every intervening task, regardless of whether those tasks are related or not.

In-order commit is very useful for reasoning about deletions and otherwise "unsynchronized" operations that would require manual synchronization to be visible. But in the case of futures, the only real problem I can see is from people writing non-idiomatic code (like checking for future readiness, which isn't safe in DCR anyway). In typical idiomatic code, it's hard to see out-of-order commit being a problem, and the performance loss is very noticeable.