explore page: use common numeraire #223

TalDerei · 2024-12-18T05:21:02Z

references #188

migrate useSummary useQuery into /trade/api,
summaries: Implements a left join of the summary with the USDC prices table in the SQL query instead of making multiple internal API server requests to /api/summary, and denominates the trading pair liquidity and volume in USDC,
stats: Implements a left join of the summary with the USDC prices table in the SQL query, and denominates the stats in USDC,
fixes internal server error in api/stats,
filters out pairs with zero liquidity and trading volume,
utility: split USDC price calculations into separate utility method

TalDerei · 2024-12-18T06:45:31Z

src/pages/trade/model/useSummary.ts

+    queries: quoteAssets.map(asset => ({
+      queryKey: ['multipleSummaries', asset, 'USDC'] as const,
+      queryFn: async () => {
+        return apiFetch<SummaryData | NoSummaryData>('/api/summary', {


It seems that the /api/stats and /api/summaries endpoints from pindexer may be returning incorrect information, impacting metrics like "total trading volume (24h)" and the largest "trading pair (24h volume)" for the USDC/USDY pair.

previous discussions suggest that this may not simply be related to the denom exponents not being handled properly. cc @cronokirby

src/pages/trade/model/useSummary.ts

src/shared/utils/price-conversion.ts

src/pages/explore/ui/pairs.tsx

src/pages/explore/api/use-summaries.ts

VanishMax

question: is it possible to implement USDC price calculation using SQL?

As I understood the code correctly, it firstly calls useSummaries, waits for it to fetch, and then makes multiple requests to /api/summary with USDC as a quote asset. In my tests, these sequential requests run from 1s to 3s. It's not a bad score, but we can sure do better. Plus, I think that making multiple requests (right now 6 /api/summary requests are made in parallel) are highly error prone and not-so-scalable.

We could change this to an improved SQL query. That's just one join with the sub-table where quote assets are USDC and multiplication method. This could also unblock sorting by calculated volume, limit and filter the results correctly.

issue (realized while writing question before): with this solution, we cannot sort the assets by usdc volume. For now, it is sorted by trade amount, which is not as useful for users as volume.

src/pages/explore/ui/pairs.tsx

src/pages/explore/ui/pair-card.tsx

src/pages/explore/ui/pairs.tsx

src/pages/trade/model/useSummary.ts

src/shared/utils/price-conversion.ts

TalDerei · 2025-01-06T18:40:41Z

note to reviewers: I'll be addressing the feedback today after a brief hiatus focusing on mobile work. @erwanor will be investigating the USDY denom coming out of pindexer.

TalDerei · 2025-01-08T06:58:24Z

question: is it possible to implement USDC price calculation using SQL?

@VanishMax coming back to this, is this something we still want to try to implementing using SQL, or is the current approach of using useMultipleSummaries sufficient? you mentioned referencing another PR that was doing something similar related to pricing?

I think we can also just iterate on this separately?

VanishMax · 2025-01-08T08:47:53Z

@TalDerei Here's the code I mentioned before: https://github.com/penumbra-zone/dex-explorer/pull/204/files#diff-389f7431dcbf5738fe1724971d0c7a329fba1bd4b7058c49dd5abbd9d875a5f6R91

I believe it's more efficient and easier to implement – the left join of the summary with usdc prices table.

cronokirby · 2025-01-08T19:19:08Z

Not a blocker for merging this, but perhaps we should instead add a column to pindexer for doing this in advance.

src/shared/api/server/stats.ts

TalDerei · 2025-01-12T21:47:47Z

src/shared/api/server/summary/pairs.ts

+      // TODO: should this be `direct_volume_over_window`?
      let volume = toValueView({
        amount: summary.liquidity,
        metadata: quoteAsset,
      });


question: should this have been summary.direct_volume_over_window rather than summary.liquidity? cc @VanishMax

i thought the stats should display the total existing liquidity (that's probably what i'd like to see on such page) but we sure can always change it

TalDerei · 2025-01-12T21:48:51Z

src/shared/utils/price-conversion.ts

+/**
+ * Calculates USDC-normalized amounts for trading pair values (volume or liquidity)
+ */
+export function calculateEquivalentInUSDC(


price calculations utility method

todo: sanity check accuracy of all consumers of calculateEquivalentInUSDC, specifically the values (volume / liquidity) and metadata being passed in.

TalDerei · 2025-01-12T22:03:02Z

src/shared/api/server/summary/types.ts

+  // Converts liquidity and trading volume to their equivalent USDC prices if `usdc_price` is available
+  if (summary.usdc_price && usdc) {
+    liquidity = calculateEquivalentInUSDC(summary.liquidity, summary.usdc_price, quoteAsset, usdc);
+
+    directVolume = calculateEquivalentInUSDC(
+      summary.direct_volume_over_window,
+      summary.usdc_price,
+      quoteAsset,
+      usdc,
+    );
+  }


todo: BTC and USDY still have strange denom coming out of pindexer – we should check if penumbra-zone/penumbra#4981 sufficiently addresses this. cc @cronokirby

pointing my local dex-explorer dev env at the prod pindexer db after conor's recent reindexing resolves this

TalDerei · 2025-01-12T22:04:08Z

src/shared/api/server/stats.ts

+      });
+
+    // Converts liquidity and trading volume to their equivalent USDC prices if `usdc_price` is available
+    if (stats.usdc_price && largestPairEnd) {


todo: api/stats not displaying correct values currently

looks more normal after recent reindexing

VanishMax

Some minor non-blocking comments. Feel free to merge

VanishMax · 2025-01-13T09:23:26Z

src/shared/utils/price-conversion.ts

+  );
+
+  // TODO: create `pnum.multiply()` utility: https://github.com/penumbra-zone/dex-explorer/issues/231
+  const result = liquidityOrVolume * usdcPrice * 10 ** expDiff;


suggestion: I see @JasonMHasperhoven using .times() function on BigNumber in collab with pnum. Maybe this could help? https://github.com/penumbra-zone/dex-explorer/blob/main/src/pages/trade/model/positions.ts#L285

src/shared/api/server/summary/single.ts

src/pages/trade/model/useMarketPrice.ts

cronokirby

This seems fine for now, and seems to work correctly based on running on a new reindex; we should probably move back some of this logic into pindexer, but we can refactor subsequently.

volume and liquidity using common numeraire (usdc)

8e660ce

TalDerei self-assigned this Dec 18, 2024

TalDerei marked this pull request as draft December 18, 2024 05:21

TalDerei commented Dec 18, 2024

View reviewed changes

TalDerei marked this pull request as ready for review December 18, 2024 06:58

VanishMax reviewed Dec 18, 2024

View reviewed changes

TalDerei commented Dec 18, 2024

View reviewed changes

src/pages/explore/ui/pairs.tsx Outdated Show resolved Hide resolved

grod220 reviewed Dec 18, 2024

View reviewed changes

TalDerei mentioned this pull request Jan 7, 2025

Add assets table to portfolio page #220

Closed

TalDerei mentioned this pull request Jan 8, 2025

create math utility class for price calculations #230

Open

TalDerei added 3 commits January 11, 2025 05:51

Merge branch 'main' into common-numeraire

409a717

explore: usdc price api using sql query

0da0a37

round down since value view expects an integer

aaf3db1

TalDerei commented Jan 12, 2025

View reviewed changes

src/shared/api/server/stats.ts Outdated Show resolved Hide resolved

TalDerei added 4 commits January 12, 2025 12:15

filter out pairs with zero liquidity and trading volume

e3f3002

linting and comments

dbf5c4a

stats: usdc-denominated stats for explore page

8af8727

utility: split price calculations into seperate utility method

4f17952

TalDerei commented Jan 12, 2025

View reviewed changes

linting and fixes

ec4fffb

TalDerei commented Jan 12, 2025

View reviewed changes

VanishMax approved these changes Jan 13, 2025

View reviewed changes

cronokirby approved these changes Jan 13, 2025

View reviewed changes

TalDerei merged commit 3687617 into main Jan 13, 2025
3 checks passed

TalDerei deleted the common-numeraire branch January 13, 2025 23:59

This was referenced Jan 14, 2025

explore page: data rendering #254

Merged

Explore portal does not render pindexer data faithfully #300

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

explore page: use common numeraire #223

explore page: use common numeraire #223

TalDerei commented Dec 18, 2024 •

edited

Loading

TalDerei Dec 18, 2024 •

edited

Loading

VanishMax left a comment

TalDerei commented Jan 6, 2025 •

edited

Loading

TalDerei commented Jan 8, 2025 •

edited

Loading

VanishMax commented Jan 8, 2025

cronokirby commented Jan 8, 2025

TalDerei Jan 12, 2025 •

edited

Loading

VanishMax Jan 13, 2025

TalDerei Jan 12, 2025 •

edited

Loading

TalDerei Jan 12, 2025 •

edited

Loading

TalDerei Jan 13, 2025

TalDerei Jan 12, 2025 •

edited

Loading

TalDerei Jan 13, 2025

VanishMax left a comment

VanishMax Jan 13, 2025

cronokirby left a comment

explore page: use common numeraire #223

explore page: use common numeraire #223

Conversation

TalDerei commented Dec 18, 2024 • edited Loading

TalDerei Dec 18, 2024 • edited Loading

Choose a reason for hiding this comment

VanishMax left a comment

Choose a reason for hiding this comment

TalDerei commented Jan 6, 2025 • edited Loading

TalDerei commented Jan 8, 2025 • edited Loading

VanishMax commented Jan 8, 2025

cronokirby commented Jan 8, 2025

TalDerei Jan 12, 2025 • edited Loading

Choose a reason for hiding this comment

VanishMax Jan 13, 2025

Choose a reason for hiding this comment

TalDerei Jan 12, 2025 • edited Loading

Choose a reason for hiding this comment

TalDerei Jan 12, 2025 • edited Loading

Choose a reason for hiding this comment

TalDerei Jan 13, 2025

Choose a reason for hiding this comment

TalDerei Jan 12, 2025 • edited Loading

Choose a reason for hiding this comment

TalDerei Jan 13, 2025

Choose a reason for hiding this comment

VanishMax left a comment

Choose a reason for hiding this comment

VanishMax Jan 13, 2025

Choose a reason for hiding this comment

cronokirby left a comment

Choose a reason for hiding this comment

TalDerei commented Dec 18, 2024 •

edited

Loading

TalDerei Dec 18, 2024 •

edited

Loading

TalDerei commented Jan 6, 2025 •

edited

Loading

TalDerei commented Jan 8, 2025 •

edited

Loading

TalDerei Jan 12, 2025 •

edited

Loading

TalDerei Jan 12, 2025 •

edited

Loading

TalDerei Jan 12, 2025 •

edited

Loading

TalDerei Jan 12, 2025 •

edited

Loading