Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ChatLSP for helpful-assistant #1492

Open
wants to merge 2 commits into
base: helpful-assistant
Choose a base branch
from
Open

ChatLSP for helpful-assistant #1492

wants to merge 2 commits into from

Conversation

disconcision
Copy link
Member

@disconcision disconcision commented Feb 1, 2025

This adds a single file, ChatLSP.re, which contains all the hazel-language-server-side internals from the llama-lsp-lookahead branch necessary to reproduce the process described in the static contextualization paper. This does not include any of the hazel frontend logic (ie prompting via ??, making the initial and followup API calls, and ensuing insertion of the response into the code).

The basic entry points are ChatLSP.Prompt.mk_init, which creates the initial prompt to be sent to the llm, and ChatLSP.Prompt.mk_error, which, given a candidate completion as a string, creates an string error report to be sent to the llm.

I've provided demos of both via the debug console (see DebugConsole.re). Pressing Shift-F9 (Fn-Shift-F9 on a macbook) will print an initial user prompt (but not the generic system prompt) relevant to the current caret position to the console:

Screenshot 2025-01-31 at 9 49 36 PM

Note that the prompt printed consists of the program sketch (here i'm just using the whole program), the expected type, relevant type definitions, and relevant function headers. When experimenting with this note that there will only be relevant defs/headers included if the expected type contains user-defined types, and there will only be relevant function headers if there are actually functions in the program which involve those types.

Pressing Shift-F10 (Fn-Shift-F10 on a macbook) will print an error report for the whole program. This will likely be useful for debugging, but note that in actual use we'll want an error report for the completion, which is slightly different in that the initial typing context and mode should reflect the cursor info of the completion location, not the generic top-level builtin initial context and synthetic mode hardcoded into this demo:

Screenshot 2025-02-01 at 3 04 05 PM

Caveats: While I cleaned up the code a bit, I brought it up to date with dev in a pretty fast-and-loose way... the type internals here will need review at some point, and some should likely be moved out of ChatLSP.re to more natural homes, but this should be more than enough to get started. While ChatLSP.re is reasonably comprehensive, many of the construction details will need to be adapted somewhat, especially when it comes to forming the program sketch, representing holes in the string representation of the program, and other assorted details of building up string/JSON messages.

I'd suggest that changes should involve defining more structured intermediary data types, and only converting things to strings/JSON on the absolute edges. For example, this version has the expected type and relevant type definitions as part of the same string. if you made a record to represent the various kind of static info, and kept stuff in the internal syntax type where appropriate, then you could use that to drive pretty-printed views of that information in the sidebar.

I've also left a variety of TODOs, some new, some old; most of these are likely not obviously actionable but are good places to look if there are bugs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant