client/core: OrderBook clarify cache usage #5
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
norwnd
commented
Mar 9, 2023
Comment on lines
+223
to
231
| // Must be done with ob.noteQueueMtx locked, otherwise some notes might be sent | ||
| // to ob.noteQueue after we are done processing it. Such notes won't be processed | ||
| // until next processCachedNotes call when they'll be irrelevant and more | ||
| // importantly it means we won't apply them on top of server snapshot they were | ||
| // meant to be applied to. | ||
| ob.setSynced(true) | ||
|
|
||
| return nil | ||
| } |
There was a problem hiding this comment.
Since cache isn't used (based on what I observed in the logs, and my current understanding of the relevant code), this fix is non-effectual really if #1 gets merged.
Moreover, we might not even need caching here really, so in the next PR I'll consider getting rid of it. But if OrderBook.noteQueue cache stays this is very much a necessary fix for it.
There was a problem hiding this comment.
Thoughts on whether or not we need (to fix or reimplement) that caching there:
- there is a 1024+128 buffer in
commspackage to hold incoming WS messages, it seems like enough of a buffer to hold those server update notifications off for a while (and probably we can easily increase these number if necessary, probably want to add someTraceflogs to monitor saturation in case it gets dangerously high during network delays and whatnot; on master I'm getting single-digits numbers fornextJobsize during normal operation, don't imagine how this could get significantly worse unless we have ~x100 activity increase on DEX maybe) - ALL server notifications will be waiting behind one that got blocked (waiting on
dc.booksMtxfor example), it doesn't seems to be a big deal here cause we don't have to handle them in real time
norwnd
force-pushed
the
master
branch
4 times, most recently
from
ad9d264 to
e298e04
Compare
norwnd
force-pushed
the
master
branch
5 times, most recently
from
f9959eb to
a66bbae
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Working on #1 I got a sense some
OrderBookstuff looks out of date. For example it hasnoteQueuethat never really gets used because update notifications can't make it there pastdc.booksMtxwhen it's locked forReset, and whendc.booksMtxis released there is no need for caching anymore (at least it should work like this with changes from #1; on master it might be used because of insufficient locking as per #1, and it's not fully atomic with respect tosynchedas is fixed in this PR).This PR mostly tries to better document those implicit assumptions we currently make about
OrderBook, and is a prerequisite ground work for the hopefully final step (which is to reduce the reliance ondc.booksMtx, effectively parallelising subscribe functionality by Market) meant to finish chain of changes #1 kicked off.