Graduate script runner (#850)

**Pull Request Checklist**
- [x] Fixes #849 
- [x] Tests still pass
- [x] Documentation/examples added
- [x] [Good commit messages](https://cbea.ms/git-commit/) and/or PR
title

**Description of PR**
Graduate the script runner out of experimental! 🚀🎓

* Removes `ExperimentalMixin` from `RunnerScriptConstructor`
* Update docs and add new "Graduated features" section to
`advanced-hera-features.md`
* Remove references to `script_runner` as experimental in docs, tests
and examples (ie. remove `global_config` accesses)

---------

Signed-off-by: Elliot Gunton <[email protected]>
Co-authored-by: Flaviu Vadan <[email protected]>

docs/examples/workflows/callable_coinflip.md

@@ -20,9 +20,6 @@
     # and serializes the output.
     global_config.image = "my-image-with-python-source-code-and-dependencies"
     global_config.set_class_defaults(Script, constructor="runner")
-    # Runner script constructor is still and experimental feature and we need to explicitly opt in to it
-    # Note that experimental features are subject to breaking changes in future releases of the same major version
-    global_config.experimental_features["script_runner"] = True
 
 
     @script()

docs/examples/workflows/callable_script.md

@@ -27,9 +27,8 @@
     # and serializes the output.
     global_config.image = "my-image-with-python-source-code-and-dependencies"
     global_config.set_class_defaults(Script, constructor="runner")
-    # Runner script constructor is still and experimental feature and we need to explicitly opt in to it
+    # Script annotations is still an experimental feature and we need to explicitly opt in to it
     # Note that experimental features are subject to breaking changes in future releases of the same major version
-    global_config.experimental_features["script_runner"] = True
     global_config.experimental_features["script_annotations"] = True
 
 

docs/examples/workflows/script_annotations_artifact_custom_volume.md

@@ -30,7 +30,6 @@ This example will reuse the outputs volume across script steps.
     )
 
     global_config.experimental_features["script_annotations"] = True
-    global_config.experimental_features["script_runner"] = True
 
 
     @script(

docs/examples/workflows/script_annotations_artifact_loaders.md

@@ -21,7 +21,6 @@
     from hera.workflows import Artifact, ArtifactLoader, Parameter, Steps, Workflow, script
 
     global_config.experimental_features["script_annotations"] = True
-    global_config.experimental_features["script_runner"] = True
 
 
     @script(constructor="runner")

docs/examples/workflows/script_annotations_artifact_outputs_defaults.md

@@ -27,7 +27,6 @@ This example will reuse the outputs volume across script steps.
     )
 
     global_config.experimental_features["script_annotations"] = True
-    global_config.experimental_features["script_runner"] = True
 
 
     @script(constructor="runner")

docs/examples/workflows/script_annotations_artifact_passing.md

@@ -27,7 +27,6 @@ This example will reuse the outputs volume across script steps.
     )
 
     global_config.experimental_features["script_annotations"] = True
-    global_config.experimental_features["script_runner"] = True
 
 
     @script(constructor="runner")

docs/examples/workflows/script_annotations_inputs.md

@@ -19,7 +19,6 @@
     from hera.workflows import Artifact, ArtifactLoader, Parameter, Steps, Workflow, script
 
     global_config.experimental_features["script_annotations"] = True
-    global_config.experimental_features["script_runner"] = True
 
 
     @script(constructor="runner")

docs/examples/workflows/script_annotations_outputs.md

@@ -20,7 +20,6 @@
     from hera.workflows import Artifact, Parameter, RunnerScriptConstructor, Steps, Workflow, script
 
     global_config.experimental_features["script_annotations"] = True
-    global_config.experimental_features["script_runner"] = True
 
     global_config.set_class_defaults(RunnerScriptConstructor, outputs_directory="/tmp/user/chosen/outputs")
 

docs/user-guides/scripts.md

@@ -138,14 +138,6 @@ as the Hera Runner runs the function by referencing it as an entrypoint of your
 should be built from the source code package itself and its dependencies, so that the source code's functions,
 dependencies, and Hera itself are available to run.
 
-The `RunnerScriptConstructor` is an experimental feature and must be enabled with the `script_runner` feature flag, as
-described in
-[the experimental features section](../walk-through/advanced-hera-features.md#experimental-features).
-
-```py
-global_config.experimental_features["script_runner"] = True
-```
-
 A function can set its `constructor` to `"runner"` to use the `RunnerScriptConstructor`, or use the
 `global_config.set_class_defaults` function to set it once for all script-decorated functions. We can write a script
 template function using Pydantic objects such as:

docs/walk-through/advanced-hera-features.md

@@ -89,7 +89,8 @@ conflicts.
 ## Experimental Features
 
 From time to time, Hera will release a new feature under the "experimental feature" flag while we develop the feature
-and ensure stability. 
+and ensure stability. Once the feature is stable and we have decided to support it long-term, it will "graduate" into
+a fully-supported feature.
 
 To enable experimental features you must set the feature by name to `True` in the `global_config.experimental_features`
 dictionary before using the feature:
@@ -98,20 +99,10 @@ dictionary before using the feature:
 global_config.experimental_features["NAME_OF_FEATURE"] = True
 ```
 
-## Currently supported experimental features:
-
-### `RunnerScriptConstructor`
-The `RunnerScriptConstructor` found in `hera.workflows.script` and seen in the
-[callable script example](../examples/workflows/callable_script.md) is a robust way to run Python functions on Argo.
-The image used by the script should be built from the source code package itself and its dependencies, so that the
-source code's functions, dependencies, and Hera itself are available to run. The `RunnerScriptConstructor` is also
-compatible with Pydantic so supports deserializing inputs to Python objects and serializing outputs to json strings. It
-must be enabled with the `script_runner` feature flag as below.
-
-```py
-global_config.experimental_features["script_runner"] = True
-```
+Note that experimental features are subject to breaking changes in future releases of the same major version. We will
+usually announce changes in [the Hera slack channel](https://cloud-native.slack.com/archives/C03NRMD9KPY).
 
+## Currently supported experimental features:
 
 ### Script Annotations
 
@@ -120,11 +111,26 @@ functions decorated as `scripts`. They use `Annotated` as the type in the functi
 writing scripts with parameters and artifacts that require additional fields such as a `description` or alternative
 `name`.
 
-
 This feature can be enabled by setting the `experimental_feature` flag `script_annotations`
 
 ```py
 global_config.experimental_features["script_annotations"] = True
 ```
 
 Read the full guide on script annotations in [the script user guide](../user-guides/scripts.md#script-annotations).
+
+## Graduated features
+
+Once an experimental feature is robust and reliable, we "graduate" them to allow their use without setting the
+`experimental_features` flag of the `global_config`. This comes with better support and guarantees for their feature
+set. We list graduated features here so you can keep up to date.
+
+### `RunnerScriptConstructor`
+
+The `RunnerScriptConstructor` found in `hera.workflows.script` and seen in the
+[callable script example](../examples/workflows/callable_script.md) is a robust way to run Python functions on Argo.
+The image used by the script should be built from the source code package itself and its dependencies, so that the
+source code's functions, dependencies, and Hera itself are available to run. The `RunnerScriptConstructor` is also
+compatible with Pydantic so supports deserializing inputs to Python objects and serializing outputs to json strings.
+
+Read [the Script Guide](../user-guides/scripts.md#runnerscriptconstructor) to learn more!

examples/workflows/callable_coinflip.py

@@ -10,9 +10,6 @@
 # and serializes the output.
 global_config.image = "my-image-with-python-source-code-and-dependencies"
 global_config.set_class_defaults(Script, constructor="runner")
-# Runner script constructor is still and experimental feature and we need to explicitly opt in to it
-# Note that experimental features are subject to breaking changes in future releases of the same major version
-global_config.experimental_features["script_runner"] = True
 
 
 @script()

examples/workflows/callable_script.py

@@ -17,9 +17,8 @@
 # and serializes the output.
 global_config.image = "my-image-with-python-source-code-and-dependencies"
 global_config.set_class_defaults(Script, constructor="runner")
-# Runner script constructor is still and experimental feature and we need to explicitly opt in to it
+# Script annotations is still an experimental feature and we need to explicitly opt in to it
 # Note that experimental features are subject to breaking changes in future releases of the same major version
-global_config.experimental_features["script_runner"] = True
 global_config.experimental_features["script_annotations"] = True
 
 

examples/workflows/script_annotations_artifact_custom_volume.py

@@ -23,7 +23,6 @@
 )
 
 global_config.experimental_features["script_annotations"] = True
-global_config.experimental_features["script_runner"] = True
 
 
 @script(

examples/workflows/script_annotations_artifact_loaders.py

@@ -11,7 +11,6 @@
 from hera.workflows import Artifact, ArtifactLoader, Parameter, Steps, Workflow, script
 
 global_config.experimental_features["script_annotations"] = True
-global_config.experimental_features["script_runner"] = True
 
 
 @script(constructor="runner")

examples/workflows/script_annotations_artifact_outputs_defaults.py

@@ -20,7 +20,6 @@
 )
 
 global_config.experimental_features["script_annotations"] = True
-global_config.experimental_features["script_runner"] = True
 
 
 @script(constructor="runner")

examples/workflows/script_annotations_artifact_passing.py

@@ -20,7 +20,6 @@
 )
 
 global_config.experimental_features["script_annotations"] = True
-global_config.experimental_features["script_runner"] = True
 
 
 @script(constructor="runner")

examples/workflows/script_annotations_inputs.py

@@ -9,7 +9,6 @@
 from hera.workflows import Artifact, ArtifactLoader, Parameter, Steps, Workflow, script
 
 global_config.experimental_features["script_annotations"] = True
-global_config.experimental_features["script_runner"] = True
 
 
 @script(constructor="runner")

examples/workflows/script_annotations_outputs.py

@@ -10,7 +10,6 @@
 from hera.workflows import Artifact, Parameter, RunnerScriptConstructor, Steps, Workflow, script
 
 global_config.experimental_features["script_annotations"] = True
-global_config.experimental_features["script_runner"] = True
 
 global_config.set_class_defaults(RunnerScriptConstructor, outputs_directory="/tmp/user/chosen/outputs")
 

src/hera/workflows/script.py

@@ -32,7 +32,6 @@
     CallableTemplateMixin,
     ContainerMixin,
     EnvIOMixin,
-    ExperimentalMixin,
     ResourceMixin,
     TemplateMixin,
     VolumeMountMixin,
@@ -692,7 +691,7 @@ def generate_source(self, instance: Script) -> str:
         return textwrap.dedent(script)
 
 
-class RunnerScriptConstructor(ScriptConstructor, ExperimentalMixin):
+class RunnerScriptConstructor(ScriptConstructor):
     """`RunnerScriptConstructor` is a script constructor that runs a script in a container.
 
     The runner script, also known as "The Hera runner", takes a script/Python function definition, infers the path
@@ -703,8 +702,6 @@ class RunnerScriptConstructor(ScriptConstructor, ExperimentalMixin):
     it conains the script to run remotely.
     """
 
-    _flag: str = "script_runner"
-
     outputs_directory: Optional[str] = None
     """Used for saving outputs when defined using annotations."""
 

tests/script_annotations/outputs.py

@@ -12,7 +12,6 @@
 from hera.workflows import Artifact, Parameter, RunnerScriptConstructor, Workflow, script
 from hera.workflows.steps import Steps
 
-global_config.experimental_features["script_runner"] = True
 global_config.experimental_features["script_annotations"] = True
 
 

tests/script_runner/annotated_outputs.py

@@ -13,7 +13,6 @@
 from hera.shared import global_config
 from hera.workflows import Artifact, Parameter, script
 
-global_config.experimental_features["script_runner"] = True
 global_config.experimental_features["script_annotations"] = True
 
 

tests/script_runner/artifact_loaders.py

@@ -13,7 +13,6 @@
 from hera.workflows.artifact import Artifact, ArtifactLoader
 
 global_config.experimental_features["script_annotations"] = True
-global_config.experimental_features["script_runner"] = True
 
 
 class MyArtifact(BaseModel):

tests/script_runner/artifact_with_invalid_loader.py

@@ -13,7 +13,6 @@
 from hera.workflows.artifact import Artifact
 
 global_config.experimental_features["script_annotations"] = True
-global_config.experimental_features["script_runner"] = True
 
 
 @script(constructor="runner")

tests/script_runner/parameter_inputs.py

@@ -11,7 +11,6 @@
 from hera.shared import global_config
 from hera.workflows import Parameter, script
 
-global_config.experimental_features["script_runner"] = True
 global_config.experimental_features["script_annotations"] = True
 
 

tests/script_runner/unknown_annotation_types.py

@@ -8,7 +8,6 @@
 from hera.workflows import script
 
 global_config.experimental_features["script_annotations"] = True
-global_config.experimental_features["script_runner"] = True
 
 
 @script(constructor="runner")

tests/test_runner.py

@@ -153,7 +153,6 @@ def test_runner_parameter_inputs(
 ):
     # GIVEN
     global_config_fixture.experimental_features["script_annotations"] = True
-    global_config_fixture.experimental_features["script_runner"] = True
 
     # WHEN
     output = _runner(entrypoint, kwargs_list)
@@ -199,7 +198,6 @@ def test_runner_annotated_parameter_inputs(
 ):
     # GIVEN
     global_config_fixture.experimental_features["script_annotations"] = True
-    global_config_fixture.experimental_features["script_runner"] = True
 
     # WHEN
     output = _runner(entrypoint, kwargs_list)
@@ -311,7 +309,6 @@ def test_script_annotations_outputs(
         assert not Path(tmp_path / file["subpath"]).is_file()
     # GIVEN
     global_config_fixture.experimental_features["script_annotations"] = True
-    global_config_fixture.experimental_features["script_runner"] = True
 
     outputs_directory = str(tmp_path / "tmp/hera/outputs")
     global_config_fixture.set_class_defaults(RunnerScriptConstructor, outputs_directory=outputs_directory)
@@ -369,7 +366,6 @@ def test_script_annotations_outputs_exceptions(
     """Test that the output annotations throw the expected exceptions."""
     # GIVEN
     global_config_fixture.experimental_features["script_annotations"] = True
-    global_config_fixture.experimental_features["script_runner"] = True
 
     # WHEN
     with pytest.raises(ValueError) as e:
@@ -472,7 +468,6 @@ def test_script_annotations_artifact_input_loader_error(
     function_name = "no_loader_invalid_type"
     kwargs_list = []
     global_config_fixture.experimental_features["script_annotations"] = True
-    global_config_fixture.experimental_features["script_runner"] = True
 
     # Force a reload of the test module, as the runner performs "importlib.import_module", which
     # may fetch a cached version