Catalog V2

[twitu]

Pull Request

Related #2062. The catalog writes data streams to parquet files. This PR ports the logic to Rust and creates pyo3 bindings for it.

• Write data streams to separate files partitioned by classes
• Query cataloged data
• Write data stream to json

Type of change

• New feature (non-breaking change which adds functionality)

How has this change been tested?

Added and ran new round trip tests

[codspeed-hq]

CodSpeed Performance Report

Merging #2071 will degrade performances by 44.59%

_{Comparing catalog-v2 (906af6c) with develop (fcb229d)}

Summary

❌ 1 regressions
✅ 51 untouched benchmarks

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

	Benchmark	develop	catalog-v2	Change
❌	test_condition_none	12.3 µs	22.1 µs	-44.59%