Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

H-3421: Resolve closed data types on creation/updating #5361

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

TimDiekmann
Copy link
Member

🌟 What is the purpose of this PR?

Data type can inherit from parent data types. This has some implications and to avoid resolving the full chain of all data types every time we need a way to resolve them when creating/updating data types.

  • description and label can be inherited from the parents as they are optional. The ClosedDataType will always inherit both at the same time if none is set on the schema itself.
  • Most importantly, the constraints needs to be iherited. Currently, this is "just" a flattened list of allOf. In a follow-up this list will be combined into a more concise representation.

🔍 What does this change?

  • Add logic to resolve closed data types
  • This does not yet expose the closed data types to the Node API nor does it cache the closed schema, it just calculates it.

Pre-Merge Checklist 🚀

🚢 Has this modified a publishable library?

This PR:

  • does not modify any publishable blocks or libraries, or modifications do not need publishing

📜 Does this require a change to the docs?

The changes in this PR:

  • are internal and do not require a docs change

🕸️ Does this require a change to the Turbo Graph?

The changes in this PR:

  • do not affect the execution graph

⚠️ Known issues

  • When calculating the closed schema it's required to sort the constraints. It's unlikely that this will even be a bottleneck, however, this could be optimized. However, we probably still want to do that to improve reading closed data types from the database.

🐾 Next steps

  • Expose get_closed_data_types to Node API
  • Create a more concise view of the constraints (hard)
  • cache the generated closed schema to the database

🛡 What tests cover this?

A test with a lot of permutations was added to make sure the types are resolved correctly in any order

@TimDiekmann TimDiekmann self-assigned this Oct 9, 2024
@github-actions github-actions bot added area/deps Relates to third-party dependencies (area) area/apps > hash* Affects HASH (a `hash-*` app) area/libs Relates to first-party libraries/crates/packages (area) type/eng > backend Owned by the @backend team area/apps area/apps > hash-graph labels Oct 9, 2024
Copy link
Contributor

github-actions bot commented Oct 9, 2024

Benchmark results

@rust/graph-benches – Integrations

representative_read_entity

Function Value Mean Flame graphs
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/page/v/2 $$17.5 \mathrm{ms} \pm 199 \mathrm{μs}\left({\color{red}25.4 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/song/v/1 $$18.1 \mathrm{ms} \pm 244 \mathrm{μs}\left({\color{red}7.54 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/person/v/1 $$17.3 \mathrm{ms} \pm 215 \mathrm{μs}\left({\color{red}6.61 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/organization/v/1 $$17.7 \mathrm{ms} \pm 216 \mathrm{μs}\left({\color{lightgreen}-26.391 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/block/v/1 $$18.0 \mathrm{ms} \pm 201 \mathrm{μs}\left({\color{red}12.4 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/uk-address/v/1 $$17.3 \mathrm{ms} \pm 223 \mathrm{μs}\left({\color{gray}-0.273 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/building/v/1 $$17.0 \mathrm{ms} \pm 181 \mathrm{μs}\left({\color{gray}-2.101 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/book/v/1 $$16.8 \mathrm{ms} \pm 168 \mathrm{μs}\left({\color{gray}-0.953 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/playlist/v/1 $$16.6 \mathrm{ms} \pm 186 \mathrm{μs}\left({\color{lightgreen}-6.401 \mathrm{\%}}\right) $$ Flame Graph

representative_read_entity_type

Function Value Mean Flame graphs
get_entity_type_by_id Account ID: d4e16033-c281-4cde-aa35-9085bf2e7579 $$1.43 \mathrm{ms} \pm 5.67 \mathrm{μs}\left({\color{gray}-1.085 \mathrm{\%}}\right) $$ Flame Graph

scaling_read_entity_complete_zero_depth

Function Value Mean Flame graphs
entity_by_id 50 entities $$4.21 \mathrm{ms} \pm 31.7 \mathrm{μs}\left({\color{gray}4.14 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 10 entities $$2.09 \mathrm{ms} \pm 11.6 \mathrm{μs}\left({\color{gray}-0.215 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 25 entities $$2.63 \mathrm{ms} \pm 16.3 \mathrm{μs}\left({\color{gray}2.83 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 1 entities $$1.91 \mathrm{ms} \pm 16.0 \mathrm{μs}\left({\color{gray}0.462 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 5 entities $$1.93 \mathrm{ms} \pm 13.8 \mathrm{μs}\left({\color{gray}0.124 \mathrm{\%}}\right) $$ Flame Graph

scaling_read_entity_linkless

Function Value Mean Flame graphs
entity_by_id 1000 entities $$2.79 \mathrm{ms} \pm 13.9 \mathrm{μs}\left({\color{gray}1.69 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 100 entities $$2.07 \mathrm{ms} \pm 6.98 \mathrm{μs}\left({\color{gray}0.966 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 10 entities $$1.90 \mathrm{ms} \pm 10.1 \mathrm{μs}\left({\color{gray}0.207 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 10000 entities $$9.41 \mathrm{ms} \pm 85.3 \mathrm{μs}\left({\color{gray}-4.493 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 1 entities $$1.90 \mathrm{ms} \pm 8.08 \mathrm{μs}\left({\color{gray}0.705 \mathrm{\%}}\right) $$ Flame Graph

representative_read_multiple_entities

Function Value Mean Flame graphs
entity_by_property depths: DT=2, PT=2, ET=2, E=2 $$60.9 \mathrm{ms} \pm 339 \mathrm{μs}\left({\color{gray}0.342 \mathrm{\%}}\right) $$ Flame Graph
entity_by_property depths: DT=255, PT=255, ET=255, E=255 $$71.1 \mathrm{ms} \pm 335 \mathrm{μs}\left({\color{gray}-0.052 \mathrm{\%}}\right) $$ Flame Graph
entity_by_property depths: DT=0, PT=0, ET=0, E=0 $$41.2 \mathrm{ms} \pm 244 \mathrm{μs}\left({\color{gray}-0.219 \mathrm{\%}}\right) $$ Flame Graph
entity_by_property depths: DT=0, PT=0, ET=0, E=2 $$45.5 \mathrm{ms} \pm 222 \mathrm{μs}\left({\color{gray}0.719 \mathrm{\%}}\right) $$ Flame Graph
entity_by_property depths: DT=0, PT=2, ET=2, E=2 $$55.8 \mathrm{ms} \pm 327 \mathrm{μs}\left({\color{gray}-2.535 \mathrm{\%}}\right) $$ Flame Graph
entity_by_property depths: DT=0, PT=0, ET=2, E=2 $$51.8 \mathrm{ms} \pm 222 \mathrm{μs}\left({\color{gray}-0.577 \mathrm{\%}}\right) $$ Flame Graph
link_by_source_by_property depths: DT=2, PT=2, ET=2, E=2 $$106 \mathrm{ms} \pm 534 \mathrm{μs}\left({\color{gray}2.08 \mathrm{\%}}\right) $$ Flame Graph
link_by_source_by_property depths: DT=255, PT=255, ET=255, E=255 $$113 \mathrm{ms} \pm 663 \mathrm{μs}\left({\color{gray}1.73 \mathrm{\%}}\right) $$ Flame Graph
link_by_source_by_property depths: DT=0, PT=0, ET=0, E=0 $$44.3 \mathrm{ms} \pm 270 \mathrm{μs}\left({\color{gray}0.275 \mathrm{\%}}\right) $$ Flame Graph
link_by_source_by_property depths: DT=0, PT=0, ET=0, E=2 $$85.3 \mathrm{ms} \pm 293 \mathrm{μs}\left({\color{gray}2.11 \mathrm{\%}}\right) $$ Flame Graph
link_by_source_by_property depths: DT=0, PT=2, ET=2, E=2 $$100 \mathrm{ms} \pm 394 \mathrm{μs}\left({\color{gray}1.48 \mathrm{\%}}\right) $$ Flame Graph
link_by_source_by_property depths: DT=0, PT=0, ET=2, E=2 $$95.0 \mathrm{ms} \pm 531 \mathrm{μs}\left({\color{gray}1.93 \mathrm{\%}}\right) $$ Flame Graph

scaling_read_entity_complete_one_depth

Function Value Mean Flame graphs
entity_by_id 50 entities $$277 \mathrm{ms} \pm 1.92 \mathrm{ms}\left({\color{gray}-1.181 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 10 entities $$52.7 \mathrm{ms} \pm 224 \mathrm{μs}\left({\color{red}6.84 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 25 entities $$77.8 \mathrm{ms} \pm 551 \mathrm{μs}\left({\color{gray}3.16 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 1 entities $$20.6 \mathrm{ms} \pm 110 \mathrm{μs}\left({\color{gray}0.759 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 5 entities $$25.6 \mathrm{ms} \pm 205 \mathrm{μs}\left({\color{gray}-0.140 \mathrm{\%}}\right) $$ Flame Graph

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/apps > hash* Affects HASH (a `hash-*` app) area/apps > hash-graph area/apps area/deps Relates to third-party dependencies (area) area/libs Relates to first-party libraries/crates/packages (area) type/eng > backend Owned by the @backend team
Development

Successfully merging this pull request may close these issues.

1 participant