Langium and embedded languages

[ejfasting]

Hello!

I'm looking into langium as a possible solution to create a new language support for vscode and monaco editor.

In my scenario i need to create a webextension (since it must be useable from monaco) and it also requires some embedded languages which is a mix of plaintext, html, css, javascript and my custom language.

Defining my custom language looks doable, but im a little unsure what is within the capabilities of langium and what is not when it comes to embedded languages (?)

From reading the documentation and other threads here on the discussion-board it seems like langium doesnt really have anything to do with embedding languages, it is strictly for creating support for a language?
I was wondering if you could somehow create a 'top level language that also contains all embedded ones', but i might be trying to screw a screw with a hammer? I also need to leverage existing LSPs for the other languages, and i highly doubt that is something langium can help with (?)

The question is kinda open, as i want input on what langium can be used for to support my scenario, and any input/suggestion are appreciated :)

//eivind

[msujew]

Hey @ejfasting,

I was wondering if you could somehow create a 'top level language that also contains all embedded ones', but i might be trying to screw a screw with a hammer?

Yes, please don't do that, it will lead down a rabbit hole you don't want to go down. It doesn't really have to do with Langium, but rather the general approach of dealing with disjointed super languages.

You probably have seen vscode's documentation on embedded languages. I would recommend something similar as they do in the Request Forwarding section. The main steps to take would be:

1. Identify the language of each document block. Send the content of each block to the corresponding language servers on document change/open. Use a specifically crafted URI for each block (something like <original-document-uri>#<block-id>, which vscode also uses for code cells of notebook documents). This way, the language server can differentiate between different blocks inside of the same document.
2. Whenever a LSP request hits the client, use a middleware to identify which language server the current block belongs to.
3. Transform the request information (i.e. adjust range/position information) and delegate the request to the appropriate language server.
4. Accept the request result and transform it back. Then return the result to vscode.

An alternative to the request information transformation would be whitespace padding. I.e. replace all data before the code block to whitespace and remove all data after the code block. That way, range/position information stays accurate across document blocks.

Basically, it all comes down to writing a lot of language client middleware, this isn't necessarily specific for Langium.

Note that the language services way of doing it could also be adapted to Langium almost 1:1 as it is outlined in the documentation. But it assumes complete control over all languages that you want include (including the top language). Since you want to include HTML, CSS and JS, I don't think this will work well unless someone has those language services available for consumption (which might be possible).

[msujew]

You should be able to run your langium-based language without creating a new language server. You can just use your language module (created from create<language>Services and use the contained services to answer the LSP requests that are coming to your volar language server. Kind of similar to how the CLI uses Langium.

[ejfasting]

aha, I didnt consider that, I'll give it a shot and update the thread (if anyone else sumbles upon this down the line).

Thanks again! :)

[ejfasting]

Update on this:
It looks like you are correct, I should be able to use the returned services to compute the completionItems. But, langium requires a LangiumDocument to be passed inn to the getCompletion-method (myservice.lsp.completionProvider.getCompletion).

What i have to work with is the TextDocument, position and token, but from what i've found you are not actually using anything else then document and position when calculating the completionItems (which is the first step to get working).

I could possibly create a langiumDocument myself, based on the interface, but it contains other properties that are mandatory. I'm unsure if it would matter that i just create dummy-data in the other properties or if i have to create this langiumDocument propertly to not break stuff.

Any advice?

[msujew]

It should be pretty simple to just use the shared.workspace.LangiumDocumentFactory to create a document from a text source.

[ejfasting]

Fantastic! This looks to be working nicely :)

For future reference:

  name: 'my-service',
  create(context): ServicePluginInstance {
    return {
      async provideCompletionItems(document, position, token) {
        var langiumDocument = shared.workspace.LangiumDocumentFactory.fromTextDocument(document);
        let params = { textDocument: document, position: position };
        const result = await Suo.lsp.CompletionProvider?.getCompletion(langiumDocument, params);
        return result;
      },
    };
  },
} satisfies ServicePlugin;