Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Creating a data clip fails on indexing when it's too large #2682

Closed
sentry-io bot opened this issue Nov 15, 2024 · 2 comments · Fixed by #2949 or #2961
Closed

Creating a data clip fails on indexing when it's too large #2682

sentry-io bot opened this issue Nov 15, 2024 · 2 comments · Fixed by #2949 or #2961
Assignees
Labels
bug Newly identified bug

Comments

@sentry-io
Copy link

sentry-io bot commented Nov 15, 2024

Sentry Issue: LIGHTNING-GK

Postgrex.Error: ERROR 54000 (program_limit_exceeded) string is too long for tsvector (1621086 bytes, max 1048575 bytes)
  File "lib/ecto/adapters/sql.ex", line 1096, in Ecto.Adapters.SQL.raise_sql_call_error/1
  File "lib/ecto/repo/schema.ex", line 834, in Ecto.Repo.Schema.apply/4
  File "lib/ecto/repo/schema.ex", line 415, in anonymous fn/15 in Ecto.Repo.Schema.do_insert/4
  File "lib/lightning/runs/handlers.ex", line 326, in anonymous fn/2 in Lightning.Runs.Handlers.CompleteStep.update_step/2
  File "lib/lightning/repo.ex", line 31, in anonymous fn/1 in Lightning.Repo.transact/2
...
(3 additional frame(s) were not displayed)
@github-project-automation github-project-automation bot moved this to New Issues in v2 Nov 15, 2024
@stuartc stuartc added the bug Newly identified bug label Feb 14, 2025
@stuartc stuartc moved this from New Issues to Backlog in v2 Feb 14, 2025
@stuartc stuartc self-assigned this Feb 14, 2025
@taylordowns2000
Copy link
Member

@josephjclark , this is what's causing the elixir error in https://github.com/OpenFn/thunderbolt/issues/462 and #2943 (seemingly not DB connection/timeout.) we are prioritizing it now.

cc @theroinaochieng

Note that we must add helptext to the SEARCH bar on history explaining that we cannot search very large dataclips. (Need human friendly language about how large, and what to do instead.)

@taylordowns2000 taylordowns2000 moved this from Tech Backlog to Ready in v2 Feb 19, 2025
@midigofrank midigofrank moved this from Ready to In progress in v2 Feb 19, 2025
@theroinaochieng theroinaochieng moved this from In progress to In review in v2 Feb 20, 2025
@github-project-automation github-project-automation bot moved this from In review to Done in v2 Feb 20, 2025
@taylordowns2000
Copy link
Member

This is awesome!!! Is there another issue to modify the text on the search bar and update the docs? I think we agreed that we'd need to:

  1. Explain in the documentation why search is currently not supported for very large dataclips.
  2. Add an info hover icon to the search bar that explains to the user that they won't be able to find the dataclip they're looking for if it's bigger than ~Xmb. (I know it's hard to estimate, given the difference in index size. But we should try to be as helpful as possible.)
Image

@taylordowns2000 taylordowns2000 moved this from Done to In review in v2 Feb 23, 2025
@github-project-automation github-project-automation bot moved this from In review to Done in v2 Feb 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Newly identified bug
Projects
Status: Done
3 participants