Let languages control multi threaded access ... is not allowed for language(s) js

[JaroslavTulach]

Feature request

The Enso language implementation has managed to significantly simplify its interoperability with JavaScript in enso-org/enso#7882 thanks to OtherContextTruffleObject concept available in recent versions of GraalVM.

I am especially thankful for removal of two thousand lines in https://github.com/enso-org/enso/pull/7882/files#diff-ccad61bee766711b658faf056e1a3f9381d9a05a9537edb2ac85774d039f0f6dL89 and related files.

Is your feature request related to a problem? Please describe.

However the functionality isn't completely on par - our original code implemented a GIL to avoid:

java.lang.IllegalStateException: Multi threaded access requested by thread main but is not allowed for language(s) js.
at com.oracle.truffle.polyglot.PolyglotEngineException.illegalState(PolyglotEngineException.java:133)
at com.oracle.truffle.polyglot.PolyglotContextImpl.throwDeniedThreadAccess(PolyglotContextImpl.java:1393)
at com.oracle.truffle.polyglot.PolyglotLanguageContext.checkThreadAccess(PolyglotLanguageContext.java:300)

However there doesn't seem to be a way for us to implement such GIL now. We can live with it for now (as Enso runs single-threaded), but Enso isn't single-threaded language and we will need a solution in the future.

Describe the solution you'd like.

We'd like to control what happens instead of IllegalStateException: Multi threaded access ... is not allowed. Options:

• specify by an option that the code shall block and wait until the access is allowed rather than throwing IllegalStateException
• specify onEnter and onExit callback that would allow to synchronize access by locking ourselves
• specify Executor that would be asked (by its execute(Runnable)) method to perform all interop operations sent to OtherContextTruffleObject
• make Graal.js multi-threading ready

Describe who do you think will benefit the most.

According to @kustosz (the creator of Enso language Truffle based interpreter) the current "Multi threaded access requested by thread main but is not allowed" isn't intuitive for people embedding Truffle languages into their Java applications at all.

I'd add: The behavior may be correct, but it is the least helpful of all possible behaviors. As Enso example shows, implementing the useful behavior requires quite a lot of code.

GraalVM users, GraalVM contributors, developers of libraries and frameworks which depend on GraalVM, or somebody else?

People who embed Truffle languages will find any kind of improvement far more friendly than the current behavior.

Describe alternatives you've considered.

We can obviously return those proxy files back and implement everything ourselves. I'd however prefer not doing so.

Additional context.

Enso provides rich support for embedding other languages: foreign section specification. Using other Truffle languages easily and smoothly straightens Enso as well as Truffle!

Express whether you'd like to help contributing this feature

One can simulate the problem in Enso test suite by removing @Ignore annotation from the test that runs JavaScript in parallel, but I can work on standalone examples that would simulate the behavior using non-Enso languages as well.

The implementation in OtherContextTruffleObject is probably too complicated to be enhanced by an outsider like me. It is a core feature behind native isolates (not sure what's the real name) and it is hard to contribute to it without having access to the native isolates. Easier if @tzezula does it ;-)

[chumer]

1. specify by an option that the code shall block and wait until the access is allowed rather than throwing IllegalStateException
2. specify onEnter and onExit callback that would allow to synchronize access by locking ourselves
3. specify Executor that would be asked (by its execute(Runnable)) method to perform all interop operations sent to OtherContextTruffleObject
4. make Graal.js multi-threading ready

Option 1 and blocking is too opinionated, and deadlock prone. Many ways to do this, so I'd rather not standardize here.
Option 2 seems doable
Option 3 incurs too much overhead, as it requires to box all arguments to the runnable in a closure.
Option 4 see https://medium.com/graalvm/multi-threaded-java-javascript-language-interoperability-in-graalvm-2f19c1f9c37b

[JaroslavTulach]

2. specify onEnter and onExit callback that would allow to synchronize access by locking ourselves

Option 2 seems doable

Great. From an Enso perspective, that's be a fantastic feature to have!

[wdanilo]

@chumer thanks for the answer. Yes, option 2 would be absolutely amazing for us. Do you think it could be delivered on your end? If so, do you have any, even very approximate estimate for it? I'm asking about it, as it is kind of important topic for us :)

[chumer]

If so, do you have any, even very approximate estimate for it?

Sorry no. We accept contributions, though. The necessary parts are all open source.

[JaroslavTulach]

Is #8266 an acceptable direction?

[chumer]

Thank you! @JaroslavTulach
I had a few comments with pros and cons: #8266 (comment)
The only real blocker is that it only works for single-threaded languages as it piggy backs on the error condition.