Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

H-4106: Refactor multi-entity type retrieval to support multiple entity type batches #6627

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

TimDiekmann
Copy link
Member

🌟 What is the purpose of this PR?

This change introduces a significant refactoring of the multi-entity type retrieval process across multiple layers of the application:

  • Updated GraphQL schema to support multiple batches of entity type IDs
  • Modified type definitions and interfaces to handle multiple entity types
  • Adjusted frontend and backend code to work with the new multi-entity type retrieval pattern
  • Updated OpenAPI specification to reflect the new parameter structure
  • Implemented batch processing in Rust store implementations

🔍 What does this change?

  • Renamed methods and types from getClosedMultiEntityType to getClosedMultiEntityTypes
  • Changed entity type retrieval to support multiple batches of entity type IDs
  • Updated type signatures and response structures to return arrays of entity types
  • Modified mapping and utility functions to handle the new data structure

Pre-Merge Checklist 🚀

🚢 Has this modified a publishable library?

This PR:

  • does not modify any publishable blocks or libraries, or modifications do not need publishing

📜 Does this require a change to the docs?

The changes in this PR:

  • are internal and do not require a docs change

🕸️ Does this require a change to the Turbo Graph?

The changes in this PR:

  • do not affect the execution graph

…batches

This change introduces a significant refactoring of the multi-entity type retrieval process across multiple layers of the application:

- Updated GraphQL schema to support multiple batches of entity type IDs
- Modified type definitions and interfaces to handle multiple entity types
- Adjusted frontend and backend code to work with the new multi-entity type retrieval pattern
- Updated OpenAPI specification to reflect the new parameter structure
- Implemented batch processing in Rust store implementations

Key changes include:
- Renamed methods and types from `getClosedMultiEntityType` to `getClosedMultiEntityTypes`
- Changed entity type retrieval to support multiple batches of entity type IDs
- Updated type signatures and response structures to return arrays of entity types
- Modified mapping and utility functions to handle the new data structure
@TimDiekmann TimDiekmann requested a review from CiaranMn March 5, 2025 13:06
@TimDiekmann TimDiekmann self-assigned this Mar 5, 2025
@github-actions github-actions bot added area/apps > hash* Affects HASH (a `hash-*` app) area/apps > hash-api Affects the HASH API (app) area/libs Relates to first-party libraries/crates/packages (area) type/eng > frontend Owned by the @frontend team type/eng > backend Owned by the @backend team area/tests New or updated tests area/tests > integration New or updated integration tests area/apps labels Mar 5, 2025
Copy link

codecov bot commented Mar 5, 2025

Codecov Report

Attention: Patch coverage is 0% with 1 line in your changes missing coverage. Please review.

Project coverage is 20.80%. Comparing base (86e8b89) to head (9103b5c).
Report is 4 commits behind head on main.

Files with missing lines Patch % Lines
...-api/src/graphql/resolvers/ontology/entity-type.ts 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #6627      +/-   ##
==========================================
+ Coverage   20.78%   20.80%   +0.01%     
==========================================
  Files         557      557              
  Lines       19401    19384      -17     
  Branches     2954     2956       +2     
==========================================
  Hits         4033     4033              
+ Misses      15311    15294      -17     
  Partials       57       57              
Flag Coverage Δ
apps.hash-ai-worker-ts 1.32% <ø> (ø)
apps.hash-api 0.00% <0.00%> (ø)
local.hash-backend-utils 4.04% <ø> (ø)
local.hash-graph-sdk 0.00% <ø> (ø)
local.hash-isomorphic-utils 0.00% <ø> (ø)
local.hash-subgraph 23.50% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Contributor

github-actions bot commented Mar 6, 2025

Benchmark results

@rust/hash-graph-benches – Integrations

representative_read_entity

Function Value Mean Flame graphs
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/song/v/1 $$15.6 \mathrm{ms} \pm 173 \mathrm{μs}\left({\color{gray}-4.980 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/block/v/1 $$15.7 \mathrm{ms} \pm 156 \mathrm{μs}\left({\color{gray}1.54 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/building/v/1 $$17.1 \mathrm{ms} \pm 181 \mathrm{μs}\left({\color{lightgreen}-25.027 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/playlist/v/1 $$17.2 \mathrm{ms} \pm 187 \mathrm{μs}\left({\color{red}5.48 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/person/v/1 $$16.2 \mathrm{ms} \pm 193 \mathrm{μs}\left({\color{lightgreen}-5.074 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/page/v/2 $$16.0 \mathrm{ms} \pm 178 \mathrm{μs}\left({\color{lightgreen}-30.010 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/organization/v/1 $$16.3 \mathrm{ms} \pm 193 \mathrm{μs}\left({\color{gray}4.38 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/uk-address/v/1 $$15.0 \mathrm{ms} \pm 151 \mathrm{μs}\left({\color{gray}-4.207 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/book/v/1 $$16.1 \mathrm{ms} \pm 180 \mathrm{μs}\left({\color{gray}-1.128 \mathrm{\%}}\right) $$ Flame Graph

scaling_read_entity_complete_zero_depth

Function Value Mean Flame graphs
entity_by_id 10 entities $$2.23 \mathrm{ms} \pm 9.77 \mathrm{μs}\left({\color{gray}-0.013 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 5 entities $$2.01 \mathrm{ms} \pm 8.31 \mathrm{μs}\left({\color{gray}-0.859 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 50 entities $$4.17 \mathrm{ms} \pm 35.5 \mathrm{μs}\left({\color{gray}1.45 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 25 entities $$3.38 \mathrm{ms} \pm 10.7 \mathrm{μs}\left({\color{gray}-1.255 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 1 entities $$2.00 \mathrm{ms} \pm 6.64 \mathrm{μs}\left({\color{gray}-0.754 \mathrm{\%}}\right) $$ Flame Graph

representative_read_entity_type

Function Value Mean Flame graphs
get_entity_type_by_id Account ID: d4e16033-c281-4cde-aa35-9085bf2e7579 $$2.15 \mathrm{ms} \pm 7.19 \mathrm{μs}\left({\color{gray}0.917 \mathrm{\%}}\right) $$ Flame Graph

scaling_read_entity_linkless

Function Value Mean Flame graphs
entity_by_id 10000 entities $$9.19 \mathrm{ms} \pm 91.8 \mathrm{μs}\left({\color{lightgreen}-31.954 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 100 entities $$2.11 \mathrm{ms} \pm 9.39 \mathrm{μs}\left({\color{gray}0.027 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 10 entities $$2.04 \mathrm{ms} \pm 9.13 \mathrm{μs}\left({\color{gray}0.109 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 1000 entities $$2.94 \mathrm{ms} \pm 12.2 \mathrm{μs}\left({\color{gray}0.297 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 1 entities $$2.02 \mathrm{ms} \pm 8.04 \mathrm{μs}\left({\color{gray}0.822 \mathrm{\%}}\right) $$ Flame Graph

representative_read_multiple_entities

Function Value Mean Flame graphs
entity_by_property depths: DT=0, PT=2, ET=2, E=2 $$51.7 \mathrm{ms} \pm 236 \mathrm{μs}\left({\color{gray}1.28 \mathrm{\%}}\right) $$ Flame Graph
entity_by_property depths: DT=0, PT=0, ET=0, E=0 $$37.1 \mathrm{ms} \pm 227 \mathrm{μs}\left({\color{gray}1.37 \mathrm{\%}}\right) $$ Flame Graph
entity_by_property depths: DT=0, PT=0, ET=0, E=2 $$41.4 \mathrm{ms} \pm 234 \mathrm{μs}\left({\color{gray}1.19 \mathrm{\%}}\right) $$ Flame Graph
entity_by_property depths: DT=2, PT=2, ET=2, E=2 $$55.2 \mathrm{ms} \pm 347 \mathrm{μs}\left({\color{gray}0.461 \mathrm{\%}}\right) $$ Flame Graph
entity_by_property depths: DT=255, PT=255, ET=255, E=255 $$64.8 \mathrm{ms} \pm 426 \mathrm{μs}\left({\color{gray}1.04 \mathrm{\%}}\right) $$ Flame Graph
entity_by_property depths: DT=0, PT=0, ET=2, E=2 $$47.0 \mathrm{ms} \pm 329 \mathrm{μs}\left({\color{gray}1.20 \mathrm{\%}}\right) $$ Flame Graph
link_by_source_by_property depths: DT=0, PT=2, ET=2, E=2 $$85.1 \mathrm{ms} \pm 551 \mathrm{μs}\left({\color{gray}0.998 \mathrm{\%}}\right) $$ Flame Graph
link_by_source_by_property depths: DT=0, PT=0, ET=0, E=0 $$36.4 \mathrm{ms} \pm 317 \mathrm{μs}\left({\color{gray}0.691 \mathrm{\%}}\right) $$ Flame Graph
link_by_source_by_property depths: DT=0, PT=0, ET=0, E=2 $$71.4 \mathrm{ms} \pm 346 \mathrm{μs}\left({\color{gray}0.544 \mathrm{\%}}\right) $$ Flame Graph
link_by_source_by_property depths: DT=2, PT=2, ET=2, E=2 $$88.8 \mathrm{ms} \pm 434 \mathrm{μs}\left({\color{gray}0.530 \mathrm{\%}}\right) $$ Flame Graph
link_by_source_by_property depths: DT=255, PT=255, ET=255, E=255 $$97.2 \mathrm{ms} \pm 505 \mathrm{μs}\left({\color{gray}0.282 \mathrm{\%}}\right) $$ Flame Graph
link_by_source_by_property depths: DT=0, PT=0, ET=2, E=2 $$80.5 \mathrm{ms} \pm 523 \mathrm{μs}\left({\color{gray}0.690 \mathrm{\%}}\right) $$ Flame Graph

scaling_read_entity_complete_one_depth

Function Value Mean Flame graphs
entity_by_id 10 entities $$57.2 \mathrm{ms} \pm 231 \mathrm{μs}\left({\color{gray}-1.929 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 5 entities $$27.1 \mathrm{ms} \pm 121 \mathrm{μs}\left({\color{gray}-1.517 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 50 entities $$5.65 \mathrm{s} \pm 285 \mathrm{ms}\left({\color{gray}5.00 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 25 entities $$83.7 \mathrm{ms} \pm 615 \mathrm{μs}\left({\color{lightgreen}-52.572 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 1 entities $$20.8 \mathrm{ms} \pm 74.7 \mathrm{μs}\left({\color{gray}-1.897 \mathrm{\%}}\right) $$ Flame Graph

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/apps > hash* Affects HASH (a `hash-*` app) area/apps > hash-api Affects the HASH API (app) area/apps area/libs Relates to first-party libraries/crates/packages (area) area/tests > integration New or updated integration tests area/tests New or updated tests type/eng > backend Owned by the @backend team type/eng > frontend Owned by the @frontend team
Development

Successfully merging this pull request may close these issues.

1 participant