Allow access to caches and instrumentation from Enso

[JaroslavTulach]

There are various issues related to current implementation of language server:

• Language Server 2.0 #5419
• Investigate transfer of Suggestion DB to IDE as binary not as JSON #5570
• Akka changing its license and the need to migrate away from it
• the need to create stable (and as small as possible) API between IDE and engine/LS: What to do on version incompatibilities between IDE and the project's specified engine #7639
• AI extensions needing access to caches

While thinking about these issues I concluded that it would be better expose the core of runtime caching & instrumentation functionality to Enso programs and use it from visualizations. The plan is to create a builtin type that would allow instrumentation of certain nodes and registration of functions to receive a callback when a node or expression evaluation is started and finished. The callback would receive the actual values flowing thru the program and could control (invalidate) the runtime cache whenever needed.

Exposing these caching & instrumentation functionality would create simple, low-level API that could easily be used to emulate the current behavior and even more easily expand it when needed. Caching & instrumentation related activities shall be as simple as writing a visualization.

We need to spend some time and investigate whether this is a viable approach. Prior work however indicates this is doable and fully functional. There is GraalVM Insight which exposes the instrumentation capabilities to scripting language. We know that GraalVM Insight works with Enso, however following should be investigated:

Tasks

Preview Give feedback

1. Expose IdExecutionInstrument functionality as a builtin
2. Demonstrate Enso program can observe execution of another Enso program
3. Provide a builtin to access and control the RuntimeCaches

[JaroslavTulach]

Making Standard.Visualization Part of the IDE

In the light of #7692 discussion and to solve the #7639 problems, let's consider following: Let's make the Standard.Visualization Enso library part of the IDE.

What problem this addresses?

The IDE contains built-in JavaScript parts of the visualizations that expect certain functions in Standard.Visualization. Currently Standard.Visualization is distributed as part of engine. As a result the IDE is tight to the exact version of engine it can work with - otherwise JavaScript parts of visualization will not find the right methods.

By packaging and distributing Standard.Visualization as part of the IDE we avoid mismatch with their JavaScript counter parts. They will always work together.

What is Needed?

• the IDE needs to tell the engine about a special repository that it should download Standard.Visualization & co. from
• so that modules used by VisualizationConfiguration can access modules from the special repository

[JaroslavTulach]

Making Language Server Part of the IDE

In the light of #7692 discussion and to solve the #7639 problems, let's consider following: Let's repackage current implementation of language server as Enso library!

What problem this addresses?

Currently the language server is bundled together with the Enso engine in enso-engine bundle. That requires us to version the language server protocol properly, negotiate at the beginning of the session the version that IDE requires and the version engine provides and reject co-operation between these two, when they don't match. The user can then either downgrade the IDE or upgrade the engine or vice versa.

By packaging the language server implementation as an Enso library Standard.Language_Server and bundling it together with the IDE we will always guarantee the he language server protocol version required by the IDE is the same as the one supported by the Standard.Language_Server library.

What is Needed?

• the IDE needs to tell the engine about a special repository that it should download Standard.Language_Server from
• the IDE needs to tell the project-manager to activate/start main method of Standard.Language_Server when launching engine
• Standard.Language_Server needs access to instrumentation and caches from the engine

[4e6]

Making Language Server Part of the IDE

the IDE needs to tell the engine about a special repository that it should download Standard.Language_Server from

We already ship IDE with the project manager. We should package the language server with IDE instead of downloading it.

[enso-bot]

Jaroslav Tulach reports a new STANDUP for yesterday (2023-10-05):

Progress: - more robust instrumentor: 6352d53

• our BigInteger support landed in Python: BigInteger support in PInt oracle/graalpython#346 (comment)
• thinking about suggestion DB for Java classes: https://discord.com/channels/@me/955430343308095518/1159485967242100756
• reviews, fun with rustup, CI fighting
• discussing language server (race condition) with Hubert: Unreliable Language Server behavior in the new GUI #7898 (comment) It should be finished by 2023-10-06.

Next Day: Bugfixing, websocket polyfill

Discord

Discord - A New Way to Chat with Friends & Communities

Discord is the easiest way to communicate over voice, video, and text. Chat, hang out, and stay close with your friends and communities.

[enso-bot]

Jaroslav Tulach reports a new STANDUP for yesterday (2023-10-06):

Progress: - META-DATA fighting: #7989

• Windows fighting
• Reported META-DATA cr/lf issue: Exchange metadata between Windows & Unix complicated because of EOL #7994
• standup & reviews & emails
• discussing enso4igv CI & changelog with Michal It should be finished by 2023-10-06.

Next Day: Bugfixing, websocket polyfill