Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Supralinear complexity with UnionParser #130

Open
amrothemich opened this issue Aug 8, 2024 · 1 comment
Open

Supralinear complexity with UnionParser #130

amrothemich opened this issue Aug 8, 2024 · 1 comment

Comments

@amrothemich
Copy link

amrothemich commented Aug 8, 2024

Hi, I have found that adding items to a JSON schema (e.g. increasing the length of an enum) increases the time ~linearly. However, it seems like adding Unions of different types of nested objects to the schema increases the complexity supralinearly. e.g. I can add an enum with 3000 items for a pretty trivial slowdown but a union of 3000 classes (with just one string field) leads to an intractable slowdown.

I see the comment

        # This is a bit of a performance hit, as it means get_allowed_characters() is called twice.

but this shouldn't cause a supralinear change, right? I'm looking into this myself but any support would be appreciated.

(Also this library is amazing, thank you!!)

@noamgat
Copy link
Owner

noamgat commented Sep 3, 2024

Its probably due to the recursive nature of compute allowed tokens. This would require in depth research to understand how to improve. I don't think I will get to it in the coming days, PRs welcome :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants