Skip to content
This repository has been archived by the owner on Jan 22, 2025. It is now read-only.

refactor(experimental): graphql: revise accounts schema for data inputs #2068

Conversation

buffalojoec
Copy link
Contributor

@buffalojoec buffalojoec commented Jan 27, 2024

This commit re-works the schema to deal with encoding and dataSlice directly
on the data field, rather than an account query at the top-level.

With this particular commit, the number of RPC calls to resolve a wide range of
queries is likely increased - such as multiple data fields on programAccounts.
However, the subsequent commits in this stack will seek to remedy this.

Specifically, the next commit introduces better account resolvers, which may
lessen the RPC calls for account queries by way of cache hits, but the commit
after that will introduce the revamped batch loader, which provides significant
optimizations to the number and timing of RPC calls.

Further up the stack are more changes to ensure the other queries, like
programAccounts, get to share in the benefits of similar optimizations.

Closes #1831

@mergify mergify bot added the community label Jan 27, 2024
@mergify mergify bot requested a review from a team January 27, 2024 22:40
@buffalojoec buffalojoec force-pushed the 01-19-refactor_experimental_graphql_revise_accounts_schema_for_data_inputs branch 2 times, most recently from 1b32527 to 6fd5498 Compare January 27, 2024 23:12
@buffalojoec buffalojoec changed the base branch from master to 01-27-refactor_experimental_graphql_drop_straggler_file January 27, 2024 23:12
@buffalojoec buffalojoec force-pushed the 01-27-refactor_experimental_graphql_drop_straggler_file branch from be0de10 to 4927097 Compare January 29, 2024 16:10
@buffalojoec buffalojoec force-pushed the 01-19-refactor_experimental_graphql_revise_accounts_schema_for_data_inputs branch from 6fd5498 to 153f535 Compare January 29, 2024 16:11
@buffalojoec buffalojoec changed the base branch from 01-27-refactor_experimental_graphql_drop_straggler_file to 01-27-refactor_experimental_graphql_use_runAllTimers_in_tests January 29, 2024 16:11
@buffalojoec buffalojoec force-pushed the 01-27-refactor_experimental_graphql_use_runAllTimers_in_tests branch 2 times, most recently from 911bcfb to 6d20469 Compare January 29, 2024 19:02
@buffalojoec buffalojoec force-pushed the 01-19-refactor_experimental_graphql_revise_accounts_schema_for_data_inputs branch from 153f535 to 3df7020 Compare January 29, 2024 19:02
@buffalojoec buffalojoec force-pushed the 01-27-refactor_experimental_graphql_use_runAllTimers_in_tests branch from 6d20469 to f05f15b Compare January 29, 2024 19:33
@buffalojoec buffalojoec force-pushed the 01-19-refactor_experimental_graphql_revise_accounts_schema_for_data_inputs branch from 3df7020 to e718da9 Compare January 29, 2024 19:33
@buffalojoec buffalojoec marked this pull request as ready for review January 29, 2024 19:33
@buffalojoec buffalojoec linked an issue Jan 30, 2024 that may be closed by this pull request
@buffalojoec buffalojoec force-pushed the 01-27-refactor_experimental_graphql_use_runAllTimers_in_tests branch from f05f15b to a4345a7 Compare February 5, 2024 19:04
@buffalojoec buffalojoec force-pushed the 01-19-refactor_experimental_graphql_revise_accounts_schema_for_data_inputs branch from e718da9 to 112f2d3 Compare February 5, 2024 19:04
@buffalojoec buffalojoec changed the base branch from 01-27-refactor_experimental_graphql_use_runAllTimers_in_tests to 02-03-refactor_experimental_graphql_add_loadMany_to_loaders February 5, 2024 19:04
Copy link
Contributor

@steveluscher steveluscher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is the time to write visitors. It'll become much more tractable to handle all of the many many edge cases:

{
  coolAccount: account {
    data(encoding: BASE58)
    ... on TokenAccount {
      mint
    }
    owner {
      dataYouDoNotWantYourVisitorToProcessYet: data(encoding: BASE64)
    }
    ...SpecialAccountsFragment
    ...CompressedDataFragment
  }
}

fragment SpecialAccountsFragment on Account {
  ... on VoteAccount {
    authority
  }
}

fragment CompressedDataFragment on Account {
  compressedData: data(encoding: BASE64ZSTD) 
}

Oooh. I think I found that ‘merge the type info into the visitor’ thing I was looking for before.

https://github.com/DxCx/graphql-js/blob/async/src/language/visitor.js#L321-L351

packages/rpc-graphql/src/resolvers/account.ts Outdated Show resolved Hide resolved
packages/rpc-graphql/src/resolvers/resolve-info/account.ts Outdated Show resolved Hide resolved
packages/rpc-graphql/src/resolvers/resolve-info/account.ts Outdated Show resolved Hide resolved
packages/rpc-graphql/src/resolvers/account.ts Show resolved Hide resolved
packages/rpc-graphql/src/resolvers/account.ts Outdated Show resolved Hide resolved
@buffalojoec buffalojoec force-pushed the 02-03-refactor_experimental_graphql_add_loadMany_to_loaders branch from fbbdd5d to 3037133 Compare February 18, 2024 16:08
@buffalojoec buffalojoec force-pushed the 01-19-refactor_experimental_graphql_revise_accounts_schema_for_data_inputs branch 2 times, most recently from 0643a09 to 6b596b0 Compare February 19, 2024 01:02
Copy link
Contributor

@steveluscher steveluscher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

K, I'm suspicious that there might be a bug here somewhere because I would not expect the test with no fragment specific fields to fetch with jsonParsed encoding. I think we need a few kinds of tests, multiplied by the three ways of making subselections (fields, inline fragments, and fragment spreads).

  • Subselections that don't require a fetch at all (ie. just fetching address)
  • Subselections that require jsonParsed (ie. literally requesting __typename, subtype-specific fields like MintAccount#mint)
  • Subselections that require baseX data

…plus a combination of each.

packages/rpc-graphql/src/resolvers/resolve-info/account.ts Outdated Show resolved Hide resolved
Comment on lines +20 to +33
if (argumentNode) {
if (argumentNode.value.kind === 'EnumValue') {
if (argumentNode.value.value === 'BASE_58') {
return 'base58';
}
if (argumentNode.value.value === 'BASE_64') {
return 'base64';
}
if (argumentNode.value.value === 'BASE_64_ZSTD') {
return 'base64+zstd';
}
}
if (argumentNode.value.kind === 'Variable') {
return variableValues[argumentNode.value.name.value] as AccountLoaderArgs['encoding'];
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could always implement this with visit() rather than spelunking in value.kind. This probably applies anywhere you find yourself doing .kind.

Or not. This is fine too!

const fragmentDefinition = info.fragments[node.name.value];
const fragmentType = schema.getType(fragmentDefinition.typeCondition.name.value)?.name;
let rootType;
if (root.kind === 'Field') {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

God I wish this thing let you write AST selectors. https://medium.com/dailyjs/eslint-selectors-just-got-super-powers-402ab942def5

Then you could do things like:

visit(node, {
    'FragmentSpread > Field': (node, key, parent, path, ancestors) => {
        // Only finds `Field` nodes who are the children of `FragmentSpread` nodes.
    },
});

If only there were like a TSQuery for GraphQL? https://medium.com/@phenomnominal/easier-typescript-tooling-with-tsquery-d74f04f2b29d

@buffalojoec buffalojoec force-pushed the 02-03-refactor_experimental_graphql_add_loadMany_to_loaders branch from 3037133 to a1b1848 Compare February 22, 2024 16:03
@buffalojoec buffalojoec force-pushed the 01-19-refactor_experimental_graphql_revise_accounts_schema_for_data_inputs branch from 6b596b0 to 0790485 Compare February 22, 2024 16:03
Comment on lines +296 to +298
expect(rpc.getAccountInfo).not.toHaveBeenCalledWith('AyGCwnwxQMCqaU4ixReHt8h5W4dwmxU7eM3BEQBdWVca', {
encoding: 'jsonParsed',
});
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@steveluscher It seems to me that the resolver is working correctly. I'm wondering if, in regards to your comments, you may have overlooked the .not here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🪦 OMG I missed that. Nice work!

@buffalojoec buffalojoec force-pushed the 01-19-refactor_experimental_graphql_revise_accounts_schema_for_data_inputs branch 2 times, most recently from 8953f62 to 10aedf8 Compare February 22, 2024 19:01
Copy link
Contributor Author

buffalojoec commented Feb 22, 2024

Merge activity

  • Feb 22, 3:57 PM EST: @buffalojoec started a stack merge that includes this pull request via Graphite.
  • Feb 22, 4:01 PM EST: Graphite rebased this pull request as part of a merge.
  • Feb 22, 4:02 PM EST: @buffalojoec merged this pull request with Graphite.

@buffalojoec buffalojoec force-pushed the 02-03-refactor_experimental_graphql_add_loadMany_to_loaders branch from a1b1848 to 169b304 Compare February 22, 2024 20:58
Base automatically changed from 02-03-refactor_experimental_graphql_add_loadMany_to_loaders to master February 22, 2024 21:00
@buffalojoec buffalojoec force-pushed the 01-19-refactor_experimental_graphql_revise_accounts_schema_for_data_inputs branch from 10aedf8 to 77e1783 Compare February 22, 2024 21:00
@buffalojoec buffalojoec merged commit 2200007 into master Feb 22, 2024
3 of 5 checks passed
@buffalojoec buffalojoec deleted the 01-19-refactor_experimental_graphql_revise_accounts_schema_for_data_inputs branch February 22, 2024 21:02
Copy link
Contributor

github-actions bot commented Mar 2, 2024

🎉 This PR is included in version 1.90.1 🎉

The release is available on:

Your semantic-release bot 📦🚀

Copy link
Contributor

Because there has been no activity on this PR for 14 days since it was merged, it has been automatically locked. Please open a new issue if it requires a follow up.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 18, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Expand on Accounts schema in GraphQL
2 participants