RuntimeManagementTest is not shutting down Threads properly

[hubertp]

As revealed in the investigation related to memory leaks.

Seems like every time RuntimeManagementTest is run, we keep two threads up and running:

Not sure if this is caused by the bug in a test or in the implementation (finalizers) of ManagedResource.

[JaroslavTulach]

Possible fix to give unnamed threads some name doesn't help as this test isn't leaking the threads:

$ git diff
diff --git engine/runtime-integration-tests/src/test/scala/org/enso/interpreter/test/semantic/RuntimeManagementTest.scala engine/runtime-integration-tests/src/test/scala/org/enso/interpreter/test/semantic/RuntimeManagementTest.scala
index 876b42ab20..7f3b018f3a 100644
--- engine/runtime-integration-tests/src/test/scala/org/enso/interpreter/test/semantic/RuntimeManagementTest.scala
+++ engine/runtime-integration-tests/src/test/scala/org/enso/interpreter/test/semantic/RuntimeManagementTest.scala
@@ -47,7 +47,7 @@ class RuntimeManagementTest extends InterpreterTest {
       }
 
       def runTest(n: Int = 5): Unit = {
-        val threads = 0.until(n).map(_ => new Thread(runnable))
+        val threads = 0.until(n).map(_ => new Thread(runnable, "RuntimeManagementTest-Thread"))
         threads.foreach(_.start())
         var reportedCount = 0
         while (reportedCount < n) {

that names one of the testing threads.

[JaroslavTulach]

Seems like every time RuntimeManagementTest is run, we keep two threads up and running:

What do you mean. "Every time" it is run? I can:

sbt:runtime-integration-tests> testOnly *RuntimeManagementTest

however that runs the test only once. How do I reproduce "every time"?

[JaroslavTulach]

Taking heap dump shows the Thread-144 hanging there after RuntimeManagementTest gets executed in a sequence of tests started by

sbt:runtime-integration-tests> test

is a LanguageSystemThread:

The Thread-143 is also LanguageSystemThread. Nobody has references to them. They point to the same instance of PolyglotImpl via the polyglot field. And to two different instances of PolyglotContextImpl via the polyglotContext field.

[JaroslavTulach]

There are four tests in RuntimeManagementTest and only the last three are leaving the threads behind. The first one creates new suspicious threads:

[info] RuntimeManagementTest:
[info] Enso Code Execution
[info]   when Context is Cached
SLF4J: Attempting to load provider "org.enso.logger.TestLogProvider" specified via "slf4j.provider" system property
[info]   - should Interrupt threads through Thread#interrupt() (1 second, 812 milliseconds)
[info]   - should Automatically free managed resources !!! IGNORED !!!
[info]   - should Automatically free managed resources amongst manual closure of other managed resources !!! IGNORED !!!
[info]   - should Automatically free managed resources amongst manual takeover of other managed resources !!! IGNORED !!!
[info] Enso Code Execution
[info]   when Context is Uncached
[info]   - should Interrupt threads through Thread#interrupt() (326 milliseconds)
[info]   - should Automatically free managed resources !!! IGNORED !!!
[info]   - should Automatically free managed resources amongst manual closure of other managed resources !!! IGNORED !!!
[info]   - should Automatically free managed resources amongst manual takeover of other managed resources !!! IGNORED !!!
Found 0 suspicious threads
[info] Test run org.enso.interpreter.test.instrument.RuntimeManagementCleanupTest finished: 0 failed, 0 ignored, 1 total, 1.057s

[JaroslavTulach]

Culprit: RuntimeManagementTest doesn't close the EnsoContext!

[enso-bot]

Jaroslav Tulach reports a new STANDUP for yesterday (2024-08-26):

Progress: - note about run a node in different context: #10719 (comment)

• addressed all comments in our PR proposed for GraalVM: Allow control of throwDeniedThreadAccess via TruffleContext.threadAccessDeniedHandler oracle/graal#8266
• setting up Oracle Support: https://www.oracle.com/support/premier/
• reworked: When there are no pendingItems for a KEEP_ALIVE period, stop RequestQueue observer thread #10890
• standup & meetings It should be finished by 2024-08-28.

Premier Support | Oracle

Elevate your Oracle experience with Premier Support for risk mitigation, cost reduction, and comprehensive system protection. Get expert 24/7 service and security.

[JaroslavTulach]

Culprit: RuntimeManagementTest doesn't close the EnsoContext!

At the end

• When there are no pendingItems for a KEEP_ALIVE period, stop RequestQueue observer thread #10890

fixes the problem by finishing the threads when they become idle.

[enso-bot]

Jaroslav Tulach reports a new STANDUP for yesterday (2024-08-27):

Progress: - minimizing moving parts: 3c28bc9

• reporting issue via Oracle Support: Backport of b957ee40f302d54edf704b36120ef21fe977453a on top of vm-23.1.3 oracle/graal#9516 (comment)
• PR merged: When there are no pendingItems for a KEEP_ALIVE period, stop RequestQueue observer thread #10890
• benchmarking PR: Benchmark and speed processing of polyglot java imports up #10899
• delaying load of Java classes: f93ee21
• investigating FramePointer: Speedup EnsoRootNode.buildFrameDescriptor #10843 (comment)
• planning & standup It should be finished by 2024-08-28.

[enso-bot]

Jaroslav Tulach reports a new STANDUP for yesterday (2024-08-28):

Progress: .

• buildDescriptor PR: Provide names of local variables via FramePointerAnalysis #10906
• reported logging issue: Don't swallow nested exceptions when logging them #10908
• meeting Pavel: IR, PartialFunction, no Option, generate with AP
• startup is below 1s: Provide names of local variables via FramePointerAnalysis #10906 (comment)
• restarting benchmarks: Benchmark and speed processing of polyglot java imports up #10899 (comment)
• assert correctness: 2765faf
• 22% slowdown due to imports: https://github.com/enso-org/enso/pull/10899/files#r1734822871
• suggestions for all: https://discord.com/channels/@me/948519891961544714/1278409643479011360
• analyzing benchmarks results: Benchmark and speed processing of polyglot java imports up #10899 (comment)
• Supplier<TruffleObject> attempt: 6ee9c8f
• standups & meetings It should be finished by 2024-08-28.

GitHub

Benchmark measuring startup with import of all libraries by JaroslavTulach · Pull Request #10899 · enso-org/enso

Pull Request Description Introduces new startup "Hello World" benchmark with many imports and a performance improvement by delaying loading classes in registerPolyglotSymbol. Checklist Pl...

Discord

Discord - Group Chat That’s All Fun & Games

Discord is great for playing games and chilling with friends, or even building a worldwide community. Customize your own space to talk, play, and hang out.