Support for Python libraries like numpy (#7678)

CHANGELOG.md

@@ -923,6 +923,7 @@
 - [Only use types as State keys][7585]
 - [Allow Java Enums in case of branches][7607]
 - [Notification about the project rename action][7613]
+- [Use `numpy` & co. from Enso!][7678]
 
 [3227]: https://github.com/enso-org/enso/pull/3227
 [3248]: https://github.com/enso-org/enso/pull/3248
@@ -1057,6 +1058,7 @@
 [7585]: https://github.com/enso-org/enso/pull/7585
 [7607]: https://github.com/enso-org/enso/pull/7607
 [7613]: https://github.com/enso-org/enso/pull/7613
+[7678]: https://github.com/enso-org/enso/pull/7678
 
 # Enso 2.0.0-alpha.18 (2021-10-12)
 

docs/polyglot/README.md

@@ -24,3 +24,5 @@ It also provides language-specific documentation for the various supported
 polyglot languages. These are as follows:
 
 - [**Java:**](./java.md) Information specific to the Java polyglot bindings.
+- [**Python:**](./python.md) Information specific to the Python polyglot
+  bindings.

docs/polyglot/polyglot-bindings.md

@@ -161,22 +161,22 @@ the language-specific documentation for details.
 ## Embedded Syntax
 
 The term "Embedded Syntax" is our terminology for the ability to use foreign
-language syntaxes from directly inside `.enso` files. This system builds upon
+language syntaxes directly from inside `.enso` files. This system builds upon
 the more generic mechanisms used by the [polyglot FFI](#the-polyglot-ffi) to
 provide a truly seamless user experience.
 
 ### Embedded Syntax Usage (Syntax)
 
 A polyglot block is introduced as follows:
 
-- The `polyglot` keyword starts a block.
-- This must be followed by a language identifier (e.g. `java`).
+- The `foreign` keyword starts a block.
+- This must be followed by a language identifier (e.g. `python`).
 - After the language identifier, the remaining syntax behaves like it is an Enso
   function definition until the `=`.
-- After the `=`, the user may write their foreign code.
+- After the `=`, the user may write their foreign code as a string.
 
 ```ruby
-polyglot python concat a b =
+foreign python concat a b = """
   def concat(a, b):
     str(a) + str(b)
 ```

docs/polyglot/python.md

@@ -0,0 +1,72 @@
+---
+layout: developer-doc
+title: Polyglot Python
+category: polyglot
+tags: [polyglot, python]
+order: 4
+---
+
+# Polyglot Python
+
+This document provides practical example showing polyglot interoperability with
+Python in the runtime. Please familiarise yourself with the general operation of
+[polyglot bindings](./polyglot-bindings.md).
+
+<!-- MarkdownTOC levels="2,3" autolink="true" -->
+
+- [Polyglot Library System](#polyglot-library-system)
+- [Using Python Libraries](#using-python-libraries)
+
+<!-- /MarkdownTOC -->
+
+## Polyglot Library System
+
+There is a support for using any Python library from Enso. Steps to include
+`numpy` in a new Enso project follows:
+
+```bash
+$ enso-engine*/bin/enso --new numenso
+$ find numenso/
+numenso/
+numenso/src
+numenso/src/Main.enso
+numenso/package.yaml
+$ mkdir numenso/polyglot
+$ graalvm/bin/gu install python
+$ graalvm/bin/graalpy -m venv numenso/polyglot/python
+$ ./numenso/polyglot/python/bin/graalpy -m pip install numpy
+Successfully installed numpy-1.23.5
+```
+
+The above steps instruct Enso to create a new project in `numenso` directory.
+Then they create Python virtual environment in `numenso/polyglot/python/` dir -
+e.g. in the
+[standard location for polyglot](../distribution/packaging.md#the-polyglot-directory)
+components of an Enso project. As a last step we activate the virtual
+environment and use `pip` manager to install `numpy` library.
+
+## Using Python Libraries
+
+As soon as a library is installed into the
+[polyglot directory](#polyglot-library-system) it can be used via the
+[embedded syntax](polyglot-bindings.md#embedded-syntax):
+
+```ruby
+foreign python random_array s = """
+    import numpy
+    return numpy.random.normal(size=s)
+
+main = random_array 10
+```
+
+Let's modify the `numenso/src/Main.enso` to use `numpy.random.normal` as shown
+above. Then we can simply execute the project and obtain a `numpy` array as a
+result:
+
+```bash
+$ enso-engine*/bin/enso --run numenso
+array([-0.51884419, -0.23670113, -1.20493508, -0.86008709,  0.59403118,
+       -0.171484  , -1.19455596, -0.30096434, -0.69762239, -0.11411331])
+```
+
+The same steps can be applied to any Graal Python supported library.

engine/runner/src/main/scala/org/enso/runner/ContextFactory.scala

@@ -8,7 +8,7 @@ import org.enso.polyglot.debugger.{
 import org.enso.polyglot.{HostAccessFactory, PolyglotContext, RuntimeOptions}
 import org.graalvm.polyglot.Context
 
-import java.io.{InputStream, OutputStream}
+import java.io.{File, InputStream, OutputStream}
 
 /** Utility class for creating Graal polyglot contexts.
   */
@@ -49,7 +49,7 @@ class ContextFactory {
     executionEnvironment.foreach { name =>
       options.put("enso.ExecutionEnvironment", name)
     }
-    val context = Context
+    val builder = Context
       .newBuilder()
       .allowExperimentalOptions(true)
       .allowAllAccess(true)
@@ -88,7 +88,16 @@ class ContextFactory {
       .logHandler(
         JavaLoggingLogHandler.create(JavaLoggingLogHandler.defaultLevelMapping)
       )
-      .build
-    new PolyglotContext(context)
+    val graalpy = new File(
+      new File(
+        new File(new File(new File(projectRoot), "polyglot"), "python"),
+        "bin"
+      ),
+      "graalpy"
+    );
+    if (graalpy.exists()) {
+      builder.option("python.Executable", graalpy.getAbsolutePath());
+    }
+    new PolyglotContext(builder.build)
   }
 }

engine/runtime-language-epb/src/main/java/org/enso/interpreter/epb/node/ForeignEvalNode.java

@@ -1,5 +1,15 @@
 package org.enso.interpreter.epb.node;
 
+import java.util.Arrays;
+import java.util.List;
+import java.util.stream.Collectors;
+
+import org.enso.interpreter.epb.EpbContext;
+import org.enso.interpreter.epb.EpbLanguage;
+import org.enso.interpreter.epb.EpbParser;
+import org.enso.interpreter.epb.runtime.ForeignParsingException;
+import org.enso.interpreter.epb.runtime.GuardedTruffleContext;
+
 import com.oracle.truffle.api.CallTarget;
 import com.oracle.truffle.api.CompilerDirectives;
 import com.oracle.truffle.api.CompilerDirectives.CompilationFinal;
@@ -10,14 +20,6 @@
 import com.oracle.truffle.api.interop.InteropLibrary;
 import com.oracle.truffle.api.nodes.RootNode;
 import com.oracle.truffle.api.source.Source;
-import java.util.Arrays;
-import java.util.List;
-import java.util.stream.Collectors;
-import org.enso.interpreter.epb.EpbContext;
-import org.enso.interpreter.epb.EpbLanguage;
-import org.enso.interpreter.epb.EpbParser;
-import org.enso.interpreter.epb.runtime.ForeignParsingException;
-import org.enso.interpreter.epb.runtime.GuardedTruffleContext;
 
 public class ForeignEvalNode extends RootNode {
   private final EpbParser.Result code;
@@ -136,12 +138,13 @@ private void parseJs() {
   private void parsePy() {
     try {
       String args = Arrays.stream(argNames).collect(Collectors.joining(","));
-      String head =
-          "import polyglot\n"
-              + "@polyglot.export_value\n"
-              + "def polyglot_enso_python_eval("
-              + args
-              + "):\n";
+      String head = """
+        import site
+        import polyglot
+        @polyglot.export_value
+        def polyglot_enso_python_eval("""
+            + args
+            + "):\n";
       String indentLines =
           code.getForeignSource().lines().map(l -> "    " + l).collect(Collectors.joining("\n"));
       Source source =