refactor(experimental): add bigint support to fast-stable-stringify

[buffalojoec]

This PR adds custom bigint support to the cache key functions used by the
resolvers within DataLoader.

The library used to create these cache keys - fast-stable-stringify - does not
support bigint, so I've added a custom cacheKey function to convert bigint
to string in all parameter objects before handing the object over to
fast-stable-stringify.

This somewhat defeats the "fast" purpose of fast-stable-stringify, but as
mentioned in the function's comments, considering the size of these parameter
objects, the effect should be negligible.

Closes #1986

[buffalojoec]

Current dependencies on/for this PR:

• master
  • PR refactor(experimental): add bigint support to fast-stable-stringify #2014 👈
    • PR refactor(experimental): fix memo instructions in GraphQL #2015
      • PR refactor(experimental): refactor GraphQL QueryArgs to LoaderArgs #2016
        • PR refactor(experimental): provide account encoding on the data field #2017

This stack of pull requests is managed by Graphite.

[steveluscher]

Oooh, hang on though. I wonder if this is our chance to fork fast-stable-stringify and just add a case to the switch (typeof val) in there, handle bigints, and get out. Thoughts?

[buffalojoec]

Oooh, hang on though. I wonder if this is our chance to fork fast-stable-stringify and just add a case to the switch (typeof val) in there, handle bigints, and get out. Thoughts?

I was close to doing that, tbh. Do we want to just put it in the Web3.js monorepo?

[buffalojoec]

In addition to the fact it hasn't had a new version in 6 years, the repository is mostly perf testing. We could probably slim it down quite a bit and add it as a package in Web3.js. Seems like it would need minimal support.

[steveluscher]

I wonder how fast an object hash would be in JS if you just walked an object depth first, turned everything you find into a byte buffer, and the called crypto.subtle.digest() on the whole thing.

function hashObject(obj) {
  const data = new Uint8Array((function*() {
    // Walk the object in the usual ways...
    Object.keys(obj).forEach(key => {
      const value = obj[key];
      switch (typeof value) {
        case 'string':
          yield* stringToBytes(value);
          break;
        case 'number': {
          const floatArray = new Float64Array(1);
          floatArray[0] = value;
          yield* new Uint8Array(floatArray.buffer);
        /* etc etc etc */
      }
    });
  })());
  const hash = await crypto.subtle.digest('SHA-256', data);
  return bytesToHex(hash);
}

[steveluscher]

It would be async though.

[steveluscher]

Somewhat related (not really though).

GoogleChromeLabs/jsbi#30 (comment)

[lorisleiva]

How about using the battle tested original json-stable-stringify package from which the "fast" one forked from?

It is downloaded over 3 million times a week and was last published 2 months ago as opposed to all the "fast" versions that were published 6 years ago. I'm pretty sure the main package would have had some extra optimisations by now.

[buffalojoec]

How about using the battle tested original json-stable-stringify package from which the "fast" one forked from?

It is downloaded over 3 million times a week and was last published 2 months ago as opposed to all the "fast" versions that were published 6 years ago. I'm pretty sure the main package would have had some extra optimisations by now.

This one looks like it might work. I can see support for setting the global BigInt.toJson or defining a replacer.

[steveluscher]

I can see support for setting the global BigInt.toJson…

Definitely definitely don't modify prototypes. web3.js will break in the first app that protects against monkeypatched builtins.

…or defining a replacer.

A replacer doesn't do what we want. What we want is this:

stringify({foo: 1n}); // {"foo":1n}

The most a replacer will let us do (demo) is this:

stringify({foo: 1n}); // {"foo":"1n"}

…which is a hash collision with this:

stringify({foo: '1n'}); // {"foo":"1n"} 

The whole root of the problem here is that we dgaf about JSON – we just need a hash – but all these libraries have as a goal the production of valid JSON.

[steveluscher]

Also, it's fucking huge. https://bundlephobia.com/package/[email protected]

[github-actions]

Because there has been no activity on this PR for 14 days since it was merged, it has been automatically locked. Please open a new issue if it requires a follow up.