"Debugging" utilities/UI

[Lotes]

I had an idea that can be implemented in several ways. I presented in the Langium Dev meeting and we decided to start a discussion here.

Usecase:
I am sometimes in the situation that I want to understand what the parser has decided during the parsing: What rules were called and what tokens were recognized by the lexer.
This information is already there (CST+AST). I just suggest some helpers to improve analysis and visualization of this information.

There are two approaches I see:

• as part of the Langium package as „DebugUtils“ or „AnalysisUtils" for example. This would be a collection of functions that can be called the during testing and for debugging purposes (maybe in the Debug Console of VS Code).
• as part of a/the Langium extension with an UI.

API-ideas:

• something like: DebugUtils.getTokens(document: LangiumDocument): Token[] will show you all leaves of the CST tree (should be a simple CST traversal, only outputting the leaves)
• Or: DebugUtils.getStacktrace(node: CstNode): AbstractRule[] will give you the decisions of the parser as a list of rules that were parsed to create this particular CST node. This can be achieved by traversing the CST node and looking at the grammarSource property.

UI-Ideas:

• We could display the AST like in the playground (JSON print or tree-like)
• For the UI I actually like the concept of hovering the text and get a tooltip of information (like here https://swift-ast-explorer.com/)
• Or even sync two views (text vs. AST vs. Stacktrace)
• We could display information like the token type and select the bounds of the token (from->to).
• We could highlight the container nodes layer for layer, upper layers are lighter (or darker)

UI-Implementation:

• The UI would require some custom JSON RPC from the language server to the user’s VSCode extension and from there to a reusable extension (see VS Code Messenger).
• if you see a better way, just throw it in the discussion :)

[spoenemann]

This is how vscode-messenger does it:

• In your own extension, export an API object including some elements to be accessed by the devtool. Example: https://github.com/TypeFox/vscode-messenger/blob/0893017c594d56f0875f90eae45d74c10fea69d4/examples/calico-colors/src/extension.ts#L56-L69
• The devtool extension gathers all compatible extensions (via vscode.extensions.all) and considers them in the contents of the provided webview: https://github.com/TypeFox/vscode-messenger/blob/main/packages/vscode-messenger-devtools/src/devtool-ext.ts