investigate higher performance serialization

[paddymul]

JSON serialization is generally pretty slow from the python side. There are also types that JS and python are capable of expressing that aren't available in JSON (infinity, NaN, NaT).

I have written half baked versions of this code many times. I would love to see a well thought out JSON serialization function not deeply tied to a larger package.

Take a look at how ipydatagrid handles this
https://github.com/bloomberg/ipydatagrid/blob/main/ipydatagrid/datagrid.py#L168-L193

When you start worrying about performance, you will want some type of binary serialization. I think parquet-wasm or arrow-js are probably the best avenues. This could handle a bunch of the edge type issues. The performance will also be much better especially from the python side. I wrote some very rough benchmarks for pandas , polars was 2-4x faster than pandas, but parquet was 100x faster.

[deanm0000]

Hey thanks for the interest.

I've got serialization improvement on my list of aspirations. For graphs that are only 1 way Python to js in a notebook is probably not going to really be noticeably slow to justify spending too much time on it. We'll see.