From 325bc2824df9bc16d46c0f74306ae1e7ae0da043 Mon Sep 17 00:00:00 2001 From: nikki everett Date: Fri, 8 Dec 2023 12:57:57 -0600 Subject: [PATCH 01/22] add basic imagespec example Signed-off-by: nikki everett --- basic-example/LICENSE | 201 ++++++++++++++++++++++++++++ basic-example/README.md | 25 ++++ basic-example/requirements.txt | 3 + basic-example/workflows/__init__.py | 0 basic-example/workflows/example.py | 70 ++++++++++ templates.json | 3 +- 6 files changed, 301 insertions(+), 1 deletion(-) create mode 100644 basic-example/LICENSE create mode 100644 basic-example/README.md create mode 100644 basic-example/requirements.txt create mode 100644 basic-example/workflows/__init__.py create mode 100644 basic-example/workflows/example.py diff --git a/basic-example/LICENSE b/basic-example/LICENSE new file mode 100644 index 0000000..f49a4e1 --- /dev/null +++ b/basic-example/LICENSE @@ -0,0 +1,201 @@ + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. \ No newline at end of file diff --git a/basic-example/README.md b/basic-example/README.md new file mode 100644 index 0000000..9f7dd42 --- /dev/null +++ b/basic-example/README.md @@ -0,0 +1,25 @@ +# {{ cookiecutter.project_name }} + +A template for the recommended layout of a Flyte enabled repository for code written in python using [flytekit](https://docs.flyte.org/projects/flytekit/en/latest/). + +## Usage + +To get up and running with your Flyte project, we recommend following the +[Flyte getting started guide](https://docs.flyte.org/en/latest/getting_started.html). + +This project includes a script `docker_build.sh` that you can use to build a +Docker image for your Flyte project. + +``` +# help +./docker_build.sh -h + +# build an image with defaults +./docker_build.sh + +# build an image with custom values +./docker_build.sh -p {{ cookiecutter.project_name }} -r -v +``` + +We recommend using a git repository to version this project, so that you can +use the git sha to version your Flyte workflows. \ No newline at end of file diff --git a/basic-example/requirements.txt b/basic-example/requirements.txt new file mode 100644 index 0000000..088b7f0 --- /dev/null +++ b/basic-example/requirements.txt @@ -0,0 +1,3 @@ +flytekit>=1.5.0 +pandas +scikit-learn \ No newline at end of file diff --git a/basic-example/workflows/__init__.py b/basic-example/workflows/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/basic-example/workflows/example.py b/basic-example/workflows/example.py new file mode 100644 index 0000000..d3780d1 --- /dev/null +++ b/basic-example/workflows/example.py @@ -0,0 +1,70 @@ +"""A simple Flyte example.""" + +import typing +from flytekit import task, workflow, ImageSpec + +""" +Image Spec is a way to specify how to build a container image without a Dockerfile. The image spec by default will be +converted to an `Envd `__ config, and the `Envd builder +`__ will build the image for you. However, you can also register your own builder to build +the image using other tools. +For every :py:class:`flytekit.PythonFunctionTask` task or a task decorated with the ``@task`` decorator, +you can specify rules for binding container images. By default, flytekit binds a single container image, i.e., +the `default Docker image `__, to all tasks. To modify this behavior, +use the ``container_image`` parameter available in the :py:func:`flytekit.task` decorator, and pass an +``ImageSpec``. +Before building the image, Flytekit checks the container registry first to see if the image already exists. By doing +so, it avoids having to rebuild the image over and over again. If the image does not exist, flytekit will build the +image before registering the workflow, and replace the image name in the task template with the newly built image name. +""" +image_definition = ImageSpec( + name="flytekit", # rename this to your docker image name + base_image="ghcr.io/flyteorg/flytekit:py3.10-1.6.0", + # this is the base image that flytekit will use to build your image + registry="ghcr.io/unionai-oss", # this is the registry where your image will be pushed to + packages=["flytekit>=1.6.0"], # these are the packages that will be installed in your image + python_version="3.10", # this is the python version that will be used to build your image +) + + +@task(container_image=image_definition) +def say_hello(name: str) -> str: + """A simple Flyte task to say "Hello". + + The @task decorator allows Flyte to use this function as a Flyte task, which + is executed as an isolated, containerized unit of compute. + """ + return f"Hello, {name}!" + + +@task(container_image=image_definition) +def greeting_length(greeting: str) -> int: + """A task the counts the length of a greeting.""" + return len(greeting) + + +@workflow +def wf(name: str = "world") -> typing.Tuple[str, int]: + """Declare workflow called `wf`. + + The @workflow decorator defines an execution graph that is composed of tasks + and potentially sub-workflows. In this simple example, the workflow is + composed of just one task. + + There are a few important things to note about workflows: + - Workflows are a domain-specific language (DSL) for creating execution + graphs and therefore only support a subset of Python's behavior. + - Tasks must be invoked with keyword arguments + - The output variables of tasks are Promises, which are placeholders for + values that are yet to be materialized, not the actual values. + """ + greeting = say_hello(name=name) + greeting_len = greeting_length(greeting=greeting) + return greeting, greeting_len + + +if __name__ == "__main__": + # Execute the workflow, simply by invoking it like a function and passing in + # the necessary parameters + print(f"Running wf() {wf(name='passengers')}") \ No newline at end of file diff --git a/templates.json b/templates.json index a6e1005..3a617e3 100644 --- a/templates.json +++ b/templates.json @@ -2,5 +2,6 @@ {"template_name": "mnist-training", "workflow_name": "mnist_workflow_cpu"}, {"template_name": "mnist-training", "workflow_name": "mnist_workflow_gpu"}, {"template_name": "simple-example", "workflow_name": "wf"}, - {"template_name": "wine-classification", "workflow_name": "training_workflow"} + {"template_name": "wine-classification", "workflow_name": "training_workflow"}, + {"template_name": "basic-example", "workflow_name": "wf"} ] From 98386cb562970b87b6c178751da79b57e77e38ca Mon Sep 17 00:00:00 2001 From: nikki everett Date: Fri, 8 Dec 2023 14:01:46 -0600 Subject: [PATCH 02/22] use latest base image and python 3.11 Signed-off-by: nikki everett --- basic-example/workflows/example.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/basic-example/workflows/example.py b/basic-example/workflows/example.py index d3780d1..0ea461e 100644 --- a/basic-example/workflows/example.py +++ b/basic-example/workflows/example.py @@ -20,11 +20,11 @@ """ image_definition = ImageSpec( name="flytekit", # rename this to your docker image name - base_image="ghcr.io/flyteorg/flytekit:py3.10-1.6.0", + base_image="ghcr.io/flyteorg/flytekit:py3.11-latest", # this is the base image that flytekit will use to build your image registry="ghcr.io/unionai-oss", # this is the registry where your image will be pushed to packages=["flytekit>=1.6.0"], # these are the packages that will be installed in your image - python_version="3.10", # this is the python version that will be used to build your image + python_version="3.11", # this is the python version that will be used to build your image ) From 887f6e9f09561949966476c4be30afef84488715 Mon Sep 17 00:00:00 2001 From: nikki everett Date: Fri, 8 Dec 2023 14:40:18 -0600 Subject: [PATCH 03/22] rename to distinguish from basic dockerfile template Signed-off-by: nikki everett --- {basic-example => basic-example-imagespec}/LICENSE | 0 {basic-example => basic-example-imagespec}/README.md | 0 {basic-example => basic-example-imagespec}/requirements.txt | 0 .../workflows/__init__.py | 0 {basic-example => basic-example-imagespec}/workflows/example.py | 0 templates.json | 2 +- 6 files changed, 1 insertion(+), 1 deletion(-) rename {basic-example => basic-example-imagespec}/LICENSE (100%) rename {basic-example => basic-example-imagespec}/README.md (100%) rename {basic-example => basic-example-imagespec}/requirements.txt (100%) rename {basic-example => basic-example-imagespec}/workflows/__init__.py (100%) rename {basic-example => basic-example-imagespec}/workflows/example.py (100%) diff --git a/basic-example/LICENSE b/basic-example-imagespec/LICENSE similarity index 100% rename from basic-example/LICENSE rename to basic-example-imagespec/LICENSE diff --git a/basic-example/README.md b/basic-example-imagespec/README.md similarity index 100% rename from basic-example/README.md rename to basic-example-imagespec/README.md diff --git a/basic-example/requirements.txt b/basic-example-imagespec/requirements.txt similarity index 100% rename from basic-example/requirements.txt rename to basic-example-imagespec/requirements.txt diff --git a/basic-example/workflows/__init__.py b/basic-example-imagespec/workflows/__init__.py similarity index 100% rename from basic-example/workflows/__init__.py rename to basic-example-imagespec/workflows/__init__.py diff --git a/basic-example/workflows/example.py b/basic-example-imagespec/workflows/example.py similarity index 100% rename from basic-example/workflows/example.py rename to basic-example-imagespec/workflows/example.py diff --git a/templates.json b/templates.json index 3a617e3..2db58ea 100644 --- a/templates.json +++ b/templates.json @@ -3,5 +3,5 @@ {"template_name": "mnist-training", "workflow_name": "mnist_workflow_gpu"}, {"template_name": "simple-example", "workflow_name": "wf"}, {"template_name": "wine-classification", "workflow_name": "training_workflow"}, - {"template_name": "basic-example", "workflow_name": "wf"} + {"template_name": "basic-example-imagespec", "workflow_name": "wf"} ] From e520fd9e3267de71d6c8e75a5fa7112b0ee2698a Mon Sep 17 00:00:00 2001 From: nikki everett Date: Fri, 8 Dec 2023 15:31:46 -0600 Subject: [PATCH 04/22] update repo README Signed-off-by: nikki everett --- README.md | 22 ++++++++++++++-------- 1 file changed, 14 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index 0e50c25..350bdd1 100644 --- a/README.md +++ b/README.md @@ -1,9 +1,16 @@ In this repository you'll find a collection of [cookiecutter](https://cookiecutter.readthedocs.io/en/latest/) templates for making [Flyte](https://github.com/flyteorg/flyte) projects. -We use cookiecutter's ability to define [multiple templates to be defined in the same repository](https://cookiecutter.readthedocs.io/en/latest/advanced/directories.html). Each cookiecutter template is defined in a separate directory, e.g. the template used in Flyte's [Getting Started](https://docs.flyte.org/en/latest/getting_started.html) guide lives in a directory called `simple-example`. +We use cookiecutter's ability to define [multiple templates to be defined in the same repository](https://cookiecutter.readthedocs.io/en/latest/advanced/directories.html). Each cookiecutter template is defined in a separate directory, e.g. the template used in Flyte's [Getting Started](https://docs.flyte.org/en/latest/getting_started.html) guide lives in a directory called `basic-example-imagespec`. -This project includes a script `docker_build.sh` that you can use to build a -Docker image for your Flyte project. +## Images + +Compiled images for all templates can be found in our [ghcr.io registry](https://github.com/flyteorg/flytekit-python-template/pkgs/container/flytekit-python-template) + +### ImageSpec vs Dockerfile + +Flytekit uses the `basic-example-imagespec` template by default when you initialize a new project with `pyflyte init`. That template uses [ImageSpec](https://docs.flyte.org/projects/cookbook/en/latest/auto_examples/customizing_dependencies/image_spec.html#image-spec-example), which builds Docker images without a Dockerfile, so doesn't contain a Dockerfile or `docker-build.sh` script. + +However, some templates in this repository contain a Dockerfile and `docker-build.sh` script that you can use to build a Docker image for your Flyte project: ``` # help @@ -16,13 +23,12 @@ Docker image for your Flyte project. ./docker_build.sh -p {{ cookiecutter.project_name }} -r -v ``` -## Images -Compiled Images for all templates can be found in our [ghcr.io registry](https://github.com/flyteorg/flytekit-python-template/pkgs/container/flytekit-python-template) +**Note:** You should only use `docker-build.sh` script to build a Docker image for a Flyte project that contains a Dockerfile. ## Usage -Each template can be rendered by specifying the `--directory` flag to cookiecutter. For example, in order to generate a project using the `simple-example` template run: +Each template can be rendered by specifying the `--directory` flag to cookiecutter. For example, in order to generate a project using the `basic-example-imagespec` template, run: - $ cookiecutter https://github.com/flyteorg/flytekit-python-template.git --directory="simple-example" + $ cookiecutter https://github.com/flyteorg/flytekit-python-template.git --directory="basic-example-imagespec" -This should prompt for a few variables that will be used to setup the project. +This should prompt for a few variables that will be used to set up the project. From ef4bbcef528746f328364886dd2562c1634a9ecb Mon Sep 17 00:00:00 2001 From: nikki everett Date: Mon, 11 Dec 2023 09:27:07 -0600 Subject: [PATCH 05/22] update imagespec Signed-off-by: nikki everett --- basic-example-imagespec/workflows/example.py | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/basic-example-imagespec/workflows/example.py b/basic-example-imagespec/workflows/example.py index 0ea461e..6bbf75d 100644 --- a/basic-example-imagespec/workflows/example.py +++ b/basic-example-imagespec/workflows/example.py @@ -19,12 +19,10 @@ image before registering the workflow, and replace the image name in the task template with the newly built image name. """ image_definition = ImageSpec( - name="flytekit", # rename this to your docker image name - base_image="ghcr.io/flyteorg/flytekit:py3.11-latest", - # this is the base image that flytekit will use to build your image - registry="ghcr.io/unionai-oss", # this is the registry where your image will be pushed to - packages=["flytekit>=1.6.0"], # these are the packages that will be installed in your image - python_version="3.11", # this is the python version that will be used to build your image + name="basic_example", # rename this to your docker image name + base_image="ghcr.io/flyteorg/flytekit:py3.11-1.10.2", + # the base image that flytekit will use to build your image + registry="ghcr.io/unionai-oss", # the registry your image will be pushed to ) From 118131276c93966de5bbab1a53ba5c91f5235bdc Mon Sep 17 00:00:00 2001 From: nikki everett Date: Mon, 11 Dec 2023 10:00:16 -0600 Subject: [PATCH 06/22] comment out imagespec definition, make linter fixes Signed-off-by: nikki everett --- basic-example-imagespec/workflows/example.py | 62 ++++++++++++-------- 1 file changed, 39 insertions(+), 23 deletions(-) diff --git a/basic-example-imagespec/workflows/example.py b/basic-example-imagespec/workflows/example.py index 6bbf75d..b9aeada 100644 --- a/basic-example-imagespec/workflows/example.py +++ b/basic-example-imagespec/workflows/example.py @@ -4,39 +4,55 @@ from flytekit import task, workflow, ImageSpec """ -Image Spec is a way to specify how to build a container image without a Dockerfile. The image spec by default will be -converted to an `Envd `__ config, and the `Envd builder +ImageSpec is a way to specify a container image configuration without a +Dockerfile. The image spec by default will be converted to an +`Envd `__ config, and the `Envd builder `__ will build the image for you. However, you can also register your own builder to build -the image using other tools. -For every :py:class:`flytekit.PythonFunctionTask` task or a task decorated with the ``@task`` decorator, -you can specify rules for binding container images. By default, flytekit binds a single container image, i.e., -the `default Docker image `__, to all tasks. To modify this behavior, -use the ``container_image`` parameter available in the :py:func:`flytekit.task` decorator, and pass an -``ImageSpec``. -Before building the image, Flytekit checks the container registry first to see if the image already exists. By doing -so, it avoids having to rebuild the image over and over again. If the image does not exist, flytekit will build the -image before registering the workflow, and replace the image name in the task template with the newly built image name. +/image_builder.py#L12-L34>`__ will build the image for you. However, you can +also register your own builder to build the image using other tools. + +For every task decorated with the ``@task`` decorator, +or :py:class:`flytekit.PythonFunctionTask` task, you can specify rules for +binding container images. By default, flytekit binds the +`default Docker image `__ to all tasks. +To modify this behavior, use the ``container_image`` parameter available +in the :py:func:`flytekit.task` decorator, and pass an ``ImageSpec``. + +Before building the image, Flytekit first checks the container registry to see +if the image already exists. By doing so, it avoids having to rebuild the +image. If the image does not exist, flytekit will build the image before +registering the workflow, and replace the image name in the task template with +the newly built image name. +""" + +# Uncomment the ImageSpec definition below and modify with your +# image definition requirements +# then set the container_image parameter on tasks that require the image """ image_definition = ImageSpec( - name="basic_example", # rename this to your docker image name + name="flytekit", # rename this to your docker image name base_image="ghcr.io/flyteorg/flytekit:py3.11-1.10.2", # the base image that flytekit will use to build your image - registry="ghcr.io/unionai-oss", # the registry your image will be pushed to + packages=[] # packages to add to the base image + registry="ghcr.io/unionai-oss", + # the registry your image will be pushed to + python_version="3.11" + # the python version, if different from the base image ) +""" -@task(container_image=image_definition) +@task() def say_hello(name: str) -> str: """A simple Flyte task to say "Hello". - The @task decorator allows Flyte to use this function as a Flyte task, which - is executed as an isolated, containerized unit of compute. + The @task decorator allows Flyte to use this function as a Flyte task, + which is executed as an isolated, containerized unit of compute. """ return f"Hello, {name}!" -@task(container_image=image_definition) +@task() def greeting_length(greeting: str) -> int: """A task the counts the length of a greeting.""" return len(greeting) @@ -46,9 +62,9 @@ def greeting_length(greeting: str) -> int: def wf(name: str = "world") -> typing.Tuple[str, int]: """Declare workflow called `wf`. - The @workflow decorator defines an execution graph that is composed of tasks - and potentially sub-workflows. In this simple example, the workflow is - composed of just one task. + The @workflow decorator defines an execution graph that is composed of + tasks and potentially sub-workflows. In this simple example, the workflow + is composed of just one task. There are a few important things to note about workflows: - Workflows are a domain-specific language (DSL) for creating execution @@ -63,6 +79,6 @@ def wf(name: str = "world") -> typing.Tuple[str, int]: if __name__ == "__main__": - # Execute the workflow, simply by invoking it like a function and passing in + # Execute the workflow by invoking it like a function and passing in # the necessary parameters - print(f"Running wf() {wf(name='passengers')}") \ No newline at end of file + print(f"Running wf() {wf(name='passengers')}") From 4a2e91af5efb85b3226d1470b0e05e1b9688f687 Mon Sep 17 00:00:00 2001 From: nikki everett Date: Mon, 11 Dec 2023 11:38:32 -0600 Subject: [PATCH 07/22] update steps Signed-off-by: nikki everett --- basic-example-imagespec/workflows/example.py | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/basic-example-imagespec/workflows/example.py b/basic-example-imagespec/workflows/example.py index b9aeada..50819e0 100644 --- a/basic-example-imagespec/workflows/example.py +++ b/basic-example-imagespec/workflows/example.py @@ -1,7 +1,7 @@ """A simple Flyte example.""" import typing -from flytekit import task, workflow, ImageSpec +from flytekit import task, workflow """ ImageSpec is a way to specify a container image configuration without a @@ -26,8 +26,9 @@ """ # Uncomment the ImageSpec definition below and modify with your -# image definition requirements -# then set the container_image parameter on tasks that require the image +# image definition requirements, +# then set the container_image parameter on tasks that require the image. +# You will also need to import ImageSpec from flytekit. """ image_definition = ImageSpec( name="flytekit", # rename this to your docker image name From fec941091860b8b8432b16ef251f7fb59bb08d1b Mon Sep 17 00:00:00 2001 From: nikki everett Date: Mon, 11 Dec 2023 12:00:51 -0600 Subject: [PATCH 08/22] make documentation more procedural Signed-off-by: nikki everett --- basic-example-imagespec/workflows/example.py | 58 +++++++------------- 1 file changed, 20 insertions(+), 38 deletions(-) diff --git a/basic-example-imagespec/workflows/example.py b/basic-example-imagespec/workflows/example.py index 50819e0..ef3495b 100644 --- a/basic-example-imagespec/workflows/example.py +++ b/basic-example-imagespec/workflows/example.py @@ -3,44 +3,26 @@ import typing from flytekit import task, workflow -""" -ImageSpec is a way to specify a container image configuration without a -Dockerfile. The image spec by default will be converted to an -`Envd `__ config, and the `Envd builder -`__ will build the image for you. However, you can -also register your own builder to build the image using other tools. - -For every task decorated with the ``@task`` decorator, -or :py:class:`flytekit.PythonFunctionTask` task, you can specify rules for -binding container images. By default, flytekit binds the -`default Docker image `__ to all tasks. -To modify this behavior, use the ``container_image`` parameter available -in the :py:func:`flytekit.task` decorator, and pass an ``ImageSpec``. - -Before building the image, Flytekit first checks the container registry to see -if the image already exists. By doing so, it avoids having to rebuild the -image. If the image does not exist, flytekit will build the image before -registering the workflow, and replace the image name in the task template with -the newly built image name. -""" - -# Uncomment the ImageSpec definition below and modify with your -# image definition requirements, -# then set the container_image parameter on tasks that require the image. -# You will also need to import ImageSpec from flytekit. -""" -image_definition = ImageSpec( - name="flytekit", # rename this to your docker image name - base_image="ghcr.io/flyteorg/flytekit:py3.11-1.10.2", - # the base image that flytekit will use to build your image - packages=[] # packages to add to the base image - registry="ghcr.io/unionai-oss", - # the registry your image will be pushed to - python_version="3.11" - # the python version, if different from the base image -) -""" +# ImageSpec is a way to specify a container image configuration without a +# Dockerfile. To use ImageSpec: +# 1. Add ImageSpec to the flytekit import line. +# 2. Uncomment the ImageSpec definition below and modify as needed. +# 3. If needed, create additional image definitions. +# 4. Set the container_image parameter on tasks that need a specific image. +# For more information, see the +# `ImageSpec documentation `__. + +# image_definition = ImageSpec( +# name="flytekit", # rename this to your docker image name +# base_image="ghcr.io/flyteorg/flytekit:py3.11-1.10.2", +# # the base image that flytekit will use to build your image +# packages=["example-package"], # packages to add to the base image +# # remove "example-package" before using. +# registry="ghcr.io/unionai-oss", +# # the registry your image will be pushed to +# python_version="3.11" +# # the python version; optional if not different from the base image +# ) @task() From 22afbdc2c9ed9ffbb962e436c15c40511ce0cb70 Mon Sep 17 00:00:00 2001 From: nikki everett Date: Mon, 11 Dec 2023 12:05:11 -0600 Subject: [PATCH 09/22] small edits Signed-off-by: nikki everett --- basic-example-imagespec/workflows/example.py | 22 ++++++++++++-------- 1 file changed, 13 insertions(+), 9 deletions(-) diff --git a/basic-example-imagespec/workflows/example.py b/basic-example-imagespec/workflows/example.py index ef3495b..cbc7730 100644 --- a/basic-example-imagespec/workflows/example.py +++ b/basic-example-imagespec/workflows/example.py @@ -3,16 +3,20 @@ import typing from flytekit import task, workflow -# ImageSpec is a way to specify a container image configuration without a -# Dockerfile. To use ImageSpec: -# 1. Add ImageSpec to the flytekit import line. -# 2. Uncomment the ImageSpec definition below and modify as needed. -# 3. If needed, create additional image definitions. -# 4. Set the container_image parameter on tasks that need a specific image. -# For more information, see the -# `ImageSpec documentation `__. -# image_definition = ImageSpec( +""" +ImageSpec is a way to specify a container image configuration without a +Dockerfile. To use ImageSpec: +1. Add ImageSpec to the flytekit import line. +2. Uncomment the ImageSpec definition below and modify as needed. +3. If needed, create additional image definitions. +4. Set the container_image parameter on tasks that need a specific image, e.g. +`@task(container_image=basic_image)` +For more information, see the +`ImageSpec documentation `__. +""" + +# basic_image = ImageSpec( # name="flytekit", # rename this to your docker image name # base_image="ghcr.io/flyteorg/flytekit:py3.11-1.10.2", # # the base image that flytekit will use to build your image From b1c54ebef48b3e2cff847e2bd0298e28689cf19e Mon Sep 17 00:00:00 2001 From: nikki everett Date: Tue, 12 Dec 2023 15:32:28 -0600 Subject: [PATCH 10/22] remove docker section Signed-off-by: nikki everett --- basic-example-imagespec/README.md | 17 ----------------- 1 file changed, 17 deletions(-) diff --git a/basic-example-imagespec/README.md b/basic-example-imagespec/README.md index 9f7dd42..8f9af81 100644 --- a/basic-example-imagespec/README.md +++ b/basic-example-imagespec/README.md @@ -6,20 +6,3 @@ A template for the recommended layout of a Flyte enabled repository for code wri To get up and running with your Flyte project, we recommend following the [Flyte getting started guide](https://docs.flyte.org/en/latest/getting_started.html). - -This project includes a script `docker_build.sh` that you can use to build a -Docker image for your Flyte project. - -``` -# help -./docker_build.sh -h - -# build an image with defaults -./docker_build.sh - -# build an image with custom values -./docker_build.sh -p {{ cookiecutter.project_name }} -r -v -``` - -We recommend using a git repository to version this project, so that you can -use the git sha to version your Flyte workflows. \ No newline at end of file From 052ed034116ef48cc0536d042c37ccf85df4a7aa Mon Sep 17 00:00:00 2001 From: nikki everett Date: Tue, 12 Dec 2023 18:51:07 -0600 Subject: [PATCH 11/22] add hello world template Signed-off-by: nikki everett --- hello-world/example.py | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) create mode 100644 hello-world/example.py diff --git a/hello-world/example.py b/hello-world/example.py new file mode 100644 index 0000000..bc6b486 --- /dev/null +++ b/hello-world/example.py @@ -0,0 +1,20 @@ +"""A basic Flyte example""" + +from flytekit import task, workflow + + +@task +def say_hello(name: str) -> str: + return f"Hello, {name}!" + + +@workflow +def hello_world_wf(name: str = 'world') -> str: + res = say_hello(name=name) + return res + + +if __name__ == "__main__": + # Execute the workflow by invoking it like a function and passing in + # the necessary parameters + print(f"Running wf() {hello_world_wf(name='passengers')}") From 34f2d8d9f465e631d24aac676d0061c388483615 Mon Sep 17 00:00:00 2001 From: nikki everett Date: Tue, 12 Dec 2023 18:53:15 -0600 Subject: [PATCH 12/22] rename directory to make it clear this is a template, remove unnecessary requirements, small edits to example python script Signed-off-by: nikki everett --- basic-template-imagespec/LICENSE | 201 ++++++++++++++++++ basic-template-imagespec/README.md | 8 + basic-template-imagespec/requirements.txt | 1 + .../workflows/__init__.py | 0 basic-template-imagespec/workflows/example.py | 71 +++++++ 5 files changed, 281 insertions(+) create mode 100644 basic-template-imagespec/LICENSE create mode 100644 basic-template-imagespec/README.md create mode 100644 basic-template-imagespec/requirements.txt create mode 100644 basic-template-imagespec/workflows/__init__.py create mode 100644 basic-template-imagespec/workflows/example.py diff --git a/basic-template-imagespec/LICENSE b/basic-template-imagespec/LICENSE new file mode 100644 index 0000000..f49a4e1 --- /dev/null +++ b/basic-template-imagespec/LICENSE @@ -0,0 +1,201 @@ + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. \ No newline at end of file diff --git a/basic-template-imagespec/README.md b/basic-template-imagespec/README.md new file mode 100644 index 0000000..8f9af81 --- /dev/null +++ b/basic-template-imagespec/README.md @@ -0,0 +1,8 @@ +# {{ cookiecutter.project_name }} + +A template for the recommended layout of a Flyte enabled repository for code written in python using [flytekit](https://docs.flyte.org/projects/flytekit/en/latest/). + +## Usage + +To get up and running with your Flyte project, we recommend following the +[Flyte getting started guide](https://docs.flyte.org/en/latest/getting_started.html). diff --git a/basic-template-imagespec/requirements.txt b/basic-template-imagespec/requirements.txt new file mode 100644 index 0000000..b1970e3 --- /dev/null +++ b/basic-template-imagespec/requirements.txt @@ -0,0 +1 @@ +flytekit>=1.5.0 \ No newline at end of file diff --git a/basic-template-imagespec/workflows/__init__.py b/basic-template-imagespec/workflows/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/basic-template-imagespec/workflows/example.py b/basic-template-imagespec/workflows/example.py new file mode 100644 index 0000000..a1d5699 --- /dev/null +++ b/basic-template-imagespec/workflows/example.py @@ -0,0 +1,71 @@ +"""A basic Flyte project template that uses ImageSpec""" + +import typing +from flytekit import task, workflow + + +""" +ImageSpec is a way to specify a container image configuration without a +Dockerfile. To use ImageSpec: +1. Add ImageSpec to the flytekit import line. +2. Uncomment the ImageSpec definition below and modify as needed. +3. If needed, create additional image definitions. +4. Set the container_image parameter on tasks that need a specific image, e.g. +`@task(container_image=basic_image)` +For more information, see the +`ImageSpec documentation `__. +""" + +# basic_image = ImageSpec( +# name="flytekit", # rename this to your docker image name +# base_image="ghcr.io/flyteorg/flytekit:py3.11-1.10.2", +# # the base image that flytekit will use to build your image +# packages=["example-package"], # packages to add to the base image +# # remove "example-package" before using. +# registry="ghcr.io/unionai-oss", +# # the registry your image will be pushed to +# python_version="3.11" +# # the python version; optional if not different from the base image +# ) + + +@task() +def say_hello(name: str) -> str: + """A simple Flyte task to say "Hello". + + The @task decorator allows Flyte to use this function as a Flyte task, + which is executed as an isolated, containerized unit of compute. + """ + return f"Hello, {name}!" + + +@task() +def greeting_length(greeting: str) -> int: + """A task the counts the length of a greeting.""" + return len(greeting) + + +@workflow +def wf(name: str = "world") -> typing.Tuple[str, int]: + """Declare workflow called `wf`. + + The @workflow decorator defines an execution graph that is composed of + tasks and potentially sub-workflows. In this simple example, the workflow + is composed of just one task. + + There are a few important things to note about workflows: + - Workflows are a domain-specific language (DSL) for creating execution + graphs and therefore only support a subset of Python's behavior. + - Tasks must be invoked with keyword arguments + - The output variables of tasks are Promises, which are placeholders for + values that are yet to be materialized, not the actual values. + """ + greeting = say_hello(name=name) + greeting_len = greeting_length(greeting=greeting) + return greeting, greeting_len + + +if __name__ == "__main__": + # Execute the workflow by invoking it like a function and passing in + # the necessary parameters + print(f"Running wf() {wf(name='passengers')}") From 82d2de3b71cf4c8f6f28095a62eae7f06d531154 Mon Sep 17 00:00:00 2001 From: nikki everett Date: Tue, 12 Dec 2023 18:54:07 -0600 Subject: [PATCH 13/22] this has been renamed Signed-off-by: nikki everett --- basic-example-imagespec/LICENSE | 201 ------------------ basic-example-imagespec/README.md | 8 - basic-example-imagespec/requirements.txt | 3 - basic-example-imagespec/workflows/__init__.py | 0 basic-example-imagespec/workflows/example.py | 71 ------- 5 files changed, 283 deletions(-) delete mode 100644 basic-example-imagespec/LICENSE delete mode 100644 basic-example-imagespec/README.md delete mode 100644 basic-example-imagespec/requirements.txt delete mode 100644 basic-example-imagespec/workflows/__init__.py delete mode 100644 basic-example-imagespec/workflows/example.py diff --git a/basic-example-imagespec/LICENSE b/basic-example-imagespec/LICENSE deleted file mode 100644 index f49a4e1..0000000 --- a/basic-example-imagespec/LICENSE +++ /dev/null @@ -1,201 +0,0 @@ - Apache License - Version 2.0, January 2004 - http://www.apache.org/licenses/ - - TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION - - 1. Definitions. - - "License" shall mean the terms and conditions for use, reproduction, - and distribution as defined by Sections 1 through 9 of this document. - - "Licensor" shall mean the copyright owner or entity authorized by - the copyright owner that is granting the License. - - "Legal Entity" shall mean the union of the acting entity and all - other entities that control, are controlled by, or are under common - control with that entity. For the purposes of this definition, - "control" means (i) the power, direct or indirect, to cause the - direction or management of such entity, whether by contract or - otherwise, or (ii) ownership of fifty percent (50%) or more of the - outstanding shares, or (iii) beneficial ownership of such entity. - - "You" (or "Your") shall mean an individual or Legal Entity - exercising permissions granted by this License. - - "Source" form shall mean the preferred form for making modifications, - including but not limited to software source code, documentation - source, and configuration files. - - "Object" form shall mean any form resulting from mechanical - transformation or translation of a Source form, including but - not limited to compiled object code, generated documentation, - and conversions to other media types. - - "Work" shall mean the work of authorship, whether in Source or - Object form, made available under the License, as indicated by a - copyright notice that is included in or attached to the work - (an example is provided in the Appendix below). - - "Derivative Works" shall mean any work, whether in Source or Object - form, that is based on (or derived from) the Work and for which the - editorial revisions, annotations, elaborations, or other modifications - represent, as a whole, an original work of authorship. For the purposes - of this License, Derivative Works shall not include works that remain - separable from, or merely link (or bind by name) to the interfaces of, - the Work and Derivative Works thereof. - - "Contribution" shall mean any work of authorship, including - the original version of the Work and any modifications or additions - to that Work or Derivative Works thereof, that is intentionally - submitted to Licensor for inclusion in the Work by the copyright owner - or by an individual or Legal Entity authorized to submit on behalf of - the copyright owner. For the purposes of this definition, "submitted" - means any form of electronic, verbal, or written communication sent - to the Licensor or its representatives, including but not limited to - communication on electronic mailing lists, source code control systems, - and issue tracking systems that are managed by, or on behalf of, the - Licensor for the purpose of discussing and improving the Work, but - excluding communication that is conspicuously marked or otherwise - designated in writing by the copyright owner as "Not a Contribution." - - "Contributor" shall mean Licensor and any individual or Legal Entity - on behalf of whom a Contribution has been received by Licensor and - subsequently incorporated within the Work. - - 2. Grant of Copyright License. Subject to the terms and conditions of - this License, each Contributor hereby grants to You a perpetual, - worldwide, non-exclusive, no-charge, royalty-free, irrevocable - copyright license to reproduce, prepare Derivative Works of, - publicly display, publicly perform, sublicense, and distribute the - Work and such Derivative Works in Source or Object form. - - 3. Grant of Patent License. Subject to the terms and conditions of - this License, each Contributor hereby grants to You a perpetual, - worldwide, non-exclusive, no-charge, royalty-free, irrevocable - (except as stated in this section) patent license to make, have made, - use, offer to sell, sell, import, and otherwise transfer the Work, - where such license applies only to those patent claims licensable - by such Contributor that are necessarily infringed by their - Contribution(s) alone or by combination of their Contribution(s) - with the Work to which such Contribution(s) was submitted. If You - institute patent litigation against any entity (including a - cross-claim or counterclaim in a lawsuit) alleging that the Work - or a Contribution incorporated within the Work constitutes direct - or contributory patent infringement, then any patent licenses - granted to You under this License for that Work shall terminate - as of the date such litigation is filed. - - 4. Redistribution. You may reproduce and distribute copies of the - Work or Derivative Works thereof in any medium, with or without - modifications, and in Source or Object form, provided that You - meet the following conditions: - - (a) You must give any other recipients of the Work or - Derivative Works a copy of this License; and - - (b) You must cause any modified files to carry prominent notices - stating that You changed the files; and - - (c) You must retain, in the Source form of any Derivative Works - that You distribute, all copyright, patent, trademark, and - attribution notices from the Source form of the Work, - excluding those notices that do not pertain to any part of - the Derivative Works; and - - (d) If the Work includes a "NOTICE" text file as part of its - distribution, then any Derivative Works that You distribute must - include a readable copy of the attribution notices contained - within such NOTICE file, excluding those notices that do not - pertain to any part of the Derivative Works, in at least one - of the following places: within a NOTICE text file distributed - as part of the Derivative Works; within the Source form or - documentation, if provided along with the Derivative Works; or, - within a display generated by the Derivative Works, if and - wherever such third-party notices normally appear. The contents - of the NOTICE file are for informational purposes only and - do not modify the License. You may add Your own attribution - notices within Derivative Works that You distribute, alongside - or as an addendum to the NOTICE text from the Work, provided - that such additional attribution notices cannot be construed - as modifying the License. - - You may add Your own copyright statement to Your modifications and - may provide additional or different license terms and conditions - for use, reproduction, or distribution of Your modifications, or - for any such Derivative Works as a whole, provided Your use, - reproduction, and distribution of the Work otherwise complies with - the conditions stated in this License. - - 5. Submission of Contributions. Unless You explicitly state otherwise, - any Contribution intentionally submitted for inclusion in the Work - by You to the Licensor shall be under the terms and conditions of - this License, without any additional terms or conditions. - Notwithstanding the above, nothing herein shall supersede or modify - the terms of any separate license agreement you may have executed - with Licensor regarding such Contributions. - - 6. Trademarks. This License does not grant permission to use the trade - names, trademarks, service marks, or product names of the Licensor, - except as required for reasonable and customary use in describing the - origin of the Work and reproducing the content of the NOTICE file. - - 7. Disclaimer of Warranty. Unless required by applicable law or - agreed to in writing, Licensor provides the Work (and each - Contributor provides its Contributions) on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or - implied, including, without limitation, any warranties or conditions - of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A - PARTICULAR PURPOSE. You are solely responsible for determining the - appropriateness of using or redistributing the Work and assume any - risks associated with Your exercise of permissions under this License. - - 8. Limitation of Liability. In no event and under no legal theory, - whether in tort (including negligence), contract, or otherwise, - unless required by applicable law (such as deliberate and grossly - negligent acts) or agreed to in writing, shall any Contributor be - liable to You for damages, including any direct, indirect, special, - incidental, or consequential damages of any character arising as a - result of this License or out of the use or inability to use the - Work (including but not limited to damages for loss of goodwill, - work stoppage, computer failure or malfunction, or any and all - other commercial damages or losses), even if such Contributor - has been advised of the possibility of such damages. - - 9. Accepting Warranty or Additional Liability. While redistributing - the Work or Derivative Works thereof, You may choose to offer, - and charge a fee for, acceptance of support, warranty, indemnity, - or other liability obligations and/or rights consistent with this - License. However, in accepting such obligations, You may act only - on Your own behalf and on Your sole responsibility, not on behalf - of any other Contributor, and only if You agree to indemnify, - defend, and hold each Contributor harmless for any liability - incurred by, or claims asserted against, such Contributor by reason - of your accepting any such warranty or additional liability. - - END OF TERMS AND CONDITIONS - - APPENDIX: How to apply the Apache License to your work. - - To apply the Apache License to your work, attach the following - boilerplate notice, with the fields enclosed by brackets "[]" - replaced with your own identifying information. (Don't include - the brackets!) The text should be enclosed in the appropriate - comment syntax for the file format. We also recommend that a - file or class name and description of purpose be included on the - same "printed page" as the copyright notice for easier - identification within third-party archives. - - Copyright [yyyy] [name of copyright owner] - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. \ No newline at end of file diff --git a/basic-example-imagespec/README.md b/basic-example-imagespec/README.md deleted file mode 100644 index 8f9af81..0000000 --- a/basic-example-imagespec/README.md +++ /dev/null @@ -1,8 +0,0 @@ -# {{ cookiecutter.project_name }} - -A template for the recommended layout of a Flyte enabled repository for code written in python using [flytekit](https://docs.flyte.org/projects/flytekit/en/latest/). - -## Usage - -To get up and running with your Flyte project, we recommend following the -[Flyte getting started guide](https://docs.flyte.org/en/latest/getting_started.html). diff --git a/basic-example-imagespec/requirements.txt b/basic-example-imagespec/requirements.txt deleted file mode 100644 index 088b7f0..0000000 --- a/basic-example-imagespec/requirements.txt +++ /dev/null @@ -1,3 +0,0 @@ -flytekit>=1.5.0 -pandas -scikit-learn \ No newline at end of file diff --git a/basic-example-imagespec/workflows/__init__.py b/basic-example-imagespec/workflows/__init__.py deleted file mode 100644 index e69de29..0000000 diff --git a/basic-example-imagespec/workflows/example.py b/basic-example-imagespec/workflows/example.py deleted file mode 100644 index cbc7730..0000000 --- a/basic-example-imagespec/workflows/example.py +++ /dev/null @@ -1,71 +0,0 @@ -"""A simple Flyte example.""" - -import typing -from flytekit import task, workflow - - -""" -ImageSpec is a way to specify a container image configuration without a -Dockerfile. To use ImageSpec: -1. Add ImageSpec to the flytekit import line. -2. Uncomment the ImageSpec definition below and modify as needed. -3. If needed, create additional image definitions. -4. Set the container_image parameter on tasks that need a specific image, e.g. -`@task(container_image=basic_image)` -For more information, see the -`ImageSpec documentation `__. -""" - -# basic_image = ImageSpec( -# name="flytekit", # rename this to your docker image name -# base_image="ghcr.io/flyteorg/flytekit:py3.11-1.10.2", -# # the base image that flytekit will use to build your image -# packages=["example-package"], # packages to add to the base image -# # remove "example-package" before using. -# registry="ghcr.io/unionai-oss", -# # the registry your image will be pushed to -# python_version="3.11" -# # the python version; optional if not different from the base image -# ) - - -@task() -def say_hello(name: str) -> str: - """A simple Flyte task to say "Hello". - - The @task decorator allows Flyte to use this function as a Flyte task, - which is executed as an isolated, containerized unit of compute. - """ - return f"Hello, {name}!" - - -@task() -def greeting_length(greeting: str) -> int: - """A task the counts the length of a greeting.""" - return len(greeting) - - -@workflow -def wf(name: str = "world") -> typing.Tuple[str, int]: - """Declare workflow called `wf`. - - The @workflow decorator defines an execution graph that is composed of - tasks and potentially sub-workflows. In this simple example, the workflow - is composed of just one task. - - There are a few important things to note about workflows: - - Workflows are a domain-specific language (DSL) for creating execution - graphs and therefore only support a subset of Python's behavior. - - Tasks must be invoked with keyword arguments - - The output variables of tasks are Promises, which are placeholders for - values that are yet to be materialized, not the actual values. - """ - greeting = say_hello(name=name) - greeting_len = greeting_length(greeting=greeting) - return greeting, greeting_len - - -if __name__ == "__main__": - # Execute the workflow by invoking it like a function and passing in - # the necessary parameters - print(f"Running wf() {wf(name='passengers')}") From 93276e9efb33f2da7bc251228274771d6b50ed55 Mon Sep 17 00:00:00 2001 From: nikki everett Date: Tue, 12 Dec 2023 19:08:37 -0600 Subject: [PATCH 14/22] update with hello world template Signed-off-by: nikki everett --- templates.json | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/templates.json b/templates.json index 2db58ea..e81b12a 100644 --- a/templates.json +++ b/templates.json @@ -3,5 +3,6 @@ {"template_name": "mnist-training", "workflow_name": "mnist_workflow_gpu"}, {"template_name": "simple-example", "workflow_name": "wf"}, {"template_name": "wine-classification", "workflow_name": "training_workflow"}, - {"template_name": "basic-example-imagespec", "workflow_name": "wf"} + {"template_name": "basic-template-imagespec", "workflow_name": "wf"}, + {"template_name": "hello_world", "workflow_name": "hello_world_wf"} ] From 66447e8345822b9f21d58ee104965189954a510e Mon Sep 17 00:00:00 2001 From: nikki everett Date: Wed, 13 Dec 2023 09:07:47 -0600 Subject: [PATCH 15/22] use cookiecutter for hello world template Signed-off-by: nikki everett --- hello-world/cookiecutter.json | 3 +++ hello-world/{ => {{cookiecutter.project_name}}}/example.py | 2 +- 2 files changed, 4 insertions(+), 1 deletion(-) create mode 100644 hello-world/cookiecutter.json rename hello-world/{ => {{cookiecutter.project_name}}}/example.py (96%) diff --git a/hello-world/cookiecutter.json b/hello-world/cookiecutter.json new file mode 100644 index 0000000..16502c8 --- /dev/null +++ b/hello-world/cookiecutter.json @@ -0,0 +1,3 @@ +{ + "project_name": "Hello world" +} diff --git a/hello-world/example.py b/hello-world/{{cookiecutter.project_name}}/example.py similarity index 96% rename from hello-world/example.py rename to hello-world/{{cookiecutter.project_name}}/example.py index bc6b486..5994819 100644 --- a/hello-world/example.py +++ b/hello-world/{{cookiecutter.project_name}}/example.py @@ -11,7 +11,7 @@ def say_hello(name: str) -> str: @workflow def hello_world_wf(name: str = 'world') -> str: res = say_hello(name=name) - return res + return res if __name__ == "__main__": From 3c4accdeb40e174bf905673fe68ffbece311c0eb Mon Sep 17 00:00:00 2001 From: nikki everett Date: Wed, 13 Dec 2023 09:10:04 -0600 Subject: [PATCH 16/22] use cookiecutter for basic imagespec template Signed-off-by: nikki everett --- basic-template-imagespec/cookiecutter.json | 3 +++ .../{ => {{cookiecuter.project_name}}}/LICENSE | 0 .../{ => {{cookiecuter.project_name}}}/README.md | 0 .../{ => {{cookiecuter.project_name}}}/requirements.txt | 0 .../{ => {{cookiecuter.project_name}}}/workflows/__init__.py | 0 .../{ => {{cookiecuter.project_name}}}/workflows/example.py | 0 6 files changed, 3 insertions(+) create mode 100644 basic-template-imagespec/cookiecutter.json rename basic-template-imagespec/{ => {{cookiecuter.project_name}}}/LICENSE (100%) rename basic-template-imagespec/{ => {{cookiecuter.project_name}}}/README.md (100%) rename basic-template-imagespec/{ => {{cookiecuter.project_name}}}/requirements.txt (100%) rename basic-template-imagespec/{ => {{cookiecuter.project_name}}}/workflows/__init__.py (100%) rename basic-template-imagespec/{ => {{cookiecuter.project_name}}}/workflows/example.py (100%) diff --git a/basic-template-imagespec/cookiecutter.json b/basic-template-imagespec/cookiecutter.json new file mode 100644 index 0000000..9634bbd --- /dev/null +++ b/basic-template-imagespec/cookiecutter.json @@ -0,0 +1,3 @@ +{ + "project_name": "Basic template ImageSpec" +} diff --git a/basic-template-imagespec/LICENSE b/basic-template-imagespec/{{cookiecuter.project_name}}/LICENSE similarity index 100% rename from basic-template-imagespec/LICENSE rename to basic-template-imagespec/{{cookiecuter.project_name}}/LICENSE diff --git a/basic-template-imagespec/README.md b/basic-template-imagespec/{{cookiecuter.project_name}}/README.md similarity index 100% rename from basic-template-imagespec/README.md rename to basic-template-imagespec/{{cookiecuter.project_name}}/README.md diff --git a/basic-template-imagespec/requirements.txt b/basic-template-imagespec/{{cookiecuter.project_name}}/requirements.txt similarity index 100% rename from basic-template-imagespec/requirements.txt rename to basic-template-imagespec/{{cookiecuter.project_name}}/requirements.txt diff --git a/basic-template-imagespec/workflows/__init__.py b/basic-template-imagespec/{{cookiecuter.project_name}}/workflows/__init__.py similarity index 100% rename from basic-template-imagespec/workflows/__init__.py rename to basic-template-imagespec/{{cookiecuter.project_name}}/workflows/__init__.py diff --git a/basic-template-imagespec/workflows/example.py b/basic-template-imagespec/{{cookiecuter.project_name}}/workflows/example.py similarity index 100% rename from basic-template-imagespec/workflows/example.py rename to basic-template-imagespec/{{cookiecuter.project_name}}/workflows/example.py From d685c3f94270d90e60a7bb460fe88e2806169698 Mon Sep 17 00:00:00 2001 From: nikki everett Date: Wed, 13 Dec 2023 10:13:26 -0600 Subject: [PATCH 17/22] small edits Signed-off-by: nikki everett --- .../{{cookiecuter.project_name}}/workflows/example.py | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/basic-template-imagespec/{{cookiecuter.project_name}}/workflows/example.py b/basic-template-imagespec/{{cookiecuter.project_name}}/workflows/example.py index a1d5699..c2e1f47 100644 --- a/basic-template-imagespec/{{cookiecuter.project_name}}/workflows/example.py +++ b/basic-template-imagespec/{{cookiecuter.project_name}}/workflows/example.py @@ -16,14 +16,12 @@ `ImageSpec documentation `__. """ -# basic_image = ImageSpec( +# image_definition = ImageSpec( # name="flytekit", # rename this to your docker image name # base_image="ghcr.io/flyteorg/flytekit:py3.11-1.10.2", # # the base image that flytekit will use to build your image -# packages=["example-package"], # packages to add to the base image -# # remove "example-package" before using. -# registry="ghcr.io/unionai-oss", -# # the registry your image will be pushed to +# packages=["flytekit>=1.6.0"], # packages to add to the base image +# registry="ghcr.io/unionai-oss", # registry your image will be pushed to # python_version="3.11" # # the python version; optional if not different from the base image # ) From a66495428474cfa87b5f6711bca5b1f2629a1c03 Mon Sep 17 00:00:00 2001 From: nikki everett Date: Wed, 13 Dec 2023 15:06:52 -0600 Subject: [PATCH 18/22] add note about default docker image Signed-off-by: nikki everett --- .../{{cookiecuter.project_name}}/workflows/example.py | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/basic-template-imagespec/{{cookiecuter.project_name}}/workflows/example.py b/basic-template-imagespec/{{cookiecuter.project_name}}/workflows/example.py index c2e1f47..cbc79b7 100644 --- a/basic-template-imagespec/{{cookiecuter.project_name}}/workflows/example.py +++ b/basic-template-imagespec/{{cookiecuter.project_name}}/workflows/example.py @@ -11,7 +11,10 @@ 2. Uncomment the ImageSpec definition below and modify as needed. 3. If needed, create additional image definitions. 4. Set the container_image parameter on tasks that need a specific image, e.g. -`@task(container_image=basic_image)` +`@task(container_image=basic_image)`. If no container_image is specified, +flytekit will use the default Docker image at +https://github.com/flyteorg/flytekit/pkgs/container/flytekit. + For more information, see the `ImageSpec documentation `__. """ From eccf0291411e7e42f2753bd6daa5fa3fd26db793 Mon Sep 17 00:00:00 2001 From: nikki everett Date: Tue, 2 Jan 2024 13:39:55 -0600 Subject: [PATCH 19/22] fix hello-world template name Signed-off-by: nikki everett --- templates.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/templates.json b/templates.json index e81b12a..0ea3f0b 100644 --- a/templates.json +++ b/templates.json @@ -4,5 +4,5 @@ {"template_name": "simple-example", "workflow_name": "wf"}, {"template_name": "wine-classification", "workflow_name": "training_workflow"}, {"template_name": "basic-template-imagespec", "workflow_name": "wf"}, - {"template_name": "hello_world", "workflow_name": "hello_world_wf"} + {"template_name": "hello-world", "workflow_name": "hello_world_wf"} ] From 57017067387e0b2e2f6addb702a8df9483c39156 Mon Sep 17 00:00:00 2001 From: nikki everett Date: Tue, 2 Jan 2024 13:41:31 -0600 Subject: [PATCH 20/22] update comments Signed-off-by: nikki everett --- .../{{cookiecuter.project_name}}/workflows/example.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/basic-template-imagespec/{{cookiecuter.project_name}}/workflows/example.py b/basic-template-imagespec/{{cookiecuter.project_name}}/workflows/example.py index cbc79b7..296817b 100644 --- a/basic-template-imagespec/{{cookiecuter.project_name}}/workflows/example.py +++ b/basic-template-imagespec/{{cookiecuter.project_name}}/workflows/example.py @@ -21,12 +21,12 @@ # image_definition = ImageSpec( # name="flytekit", # rename this to your docker image name -# base_image="ghcr.io/flyteorg/flytekit:py3.11-1.10.2", # # the base image that flytekit will use to build your image +# base_image="ghcr.io/flyteorg/flytekit:py3.11-1.10.2", # packages=["flytekit>=1.6.0"], # packages to add to the base image # registry="ghcr.io/unionai-oss", # registry your image will be pushed to -# python_version="3.11" # # the python version; optional if not different from the base image +# python_version="3.11" # ) From 3df55823184b07b0236276178d4a7bffa6048d4c Mon Sep 17 00:00:00 2001 From: nikki everett Date: Tue, 2 Jan 2024 13:43:22 -0600 Subject: [PATCH 21/22] rename image definition Signed-off-by: nikki everett --- .../{{cookiecuter.project_name}}/workflows/example.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/basic-template-imagespec/{{cookiecuter.project_name}}/workflows/example.py b/basic-template-imagespec/{{cookiecuter.project_name}}/workflows/example.py index 296817b..f4c15d8 100644 --- a/basic-template-imagespec/{{cookiecuter.project_name}}/workflows/example.py +++ b/basic-template-imagespec/{{cookiecuter.project_name}}/workflows/example.py @@ -20,7 +20,7 @@ """ # image_definition = ImageSpec( -# name="flytekit", # rename this to your docker image name +# name="basic-example", # rename this to your docker image name # # the base image that flytekit will use to build your image # base_image="ghcr.io/flyteorg/flytekit:py3.11-1.10.2", # packages=["flytekit>=1.6.0"], # packages to add to the base image From d001f54e004ed58e085760ee0a03d35e7cdd46b4 Mon Sep 17 00:00:00 2001 From: nikki everett Date: Thu, 4 Jan 2024 13:00:27 -0600 Subject: [PATCH 22/22] rename for consistency Signed-off-by: nikki everett --- README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 350bdd1..609d976 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ In this repository you'll find a collection of [cookiecutter](https://cookiecutter.readthedocs.io/en/latest/) templates for making [Flyte](https://github.com/flyteorg/flyte) projects. -We use cookiecutter's ability to define [multiple templates to be defined in the same repository](https://cookiecutter.readthedocs.io/en/latest/advanced/directories.html). Each cookiecutter template is defined in a separate directory, e.g. the template used in Flyte's [Getting Started](https://docs.flyte.org/en/latest/getting_started.html) guide lives in a directory called `basic-example-imagespec`. +We use cookiecutter's ability to define [multiple templates to be defined in the same repository](https://cookiecutter.readthedocs.io/en/latest/advanced/directories.html). Each cookiecutter template is defined in a separate directory, e.g. the template used in Flyte's [Getting Started](https://docs.flyte.org/en/latest/getting_started.html) guide lives in a directory called `basic-template-imagespec`. ## Images @@ -8,7 +8,7 @@ Compiled images for all templates can be found in our [ghcr.io registry](https:/ ### ImageSpec vs Dockerfile -Flytekit uses the `basic-example-imagespec` template by default when you initialize a new project with `pyflyte init`. That template uses [ImageSpec](https://docs.flyte.org/projects/cookbook/en/latest/auto_examples/customizing_dependencies/image_spec.html#image-spec-example), which builds Docker images without a Dockerfile, so doesn't contain a Dockerfile or `docker-build.sh` script. +Flytekit uses the `basic-template-imagespec` template by default when you initialize a new project with `pyflyte init`. That template uses [ImageSpec](https://docs.flyte.org/projects/cookbook/en/latest/auto_examples/customizing_dependencies/image_spec.html#image-spec-example), which builds Docker images without a Dockerfile, so doesn't contain a Dockerfile or `docker-build.sh` script. However, some templates in this repository contain a Dockerfile and `docker-build.sh` script that you can use to build a Docker image for your Flyte project: @@ -27,8 +27,8 @@ However, some templates in this repository contain a Dockerfile and `docker-buil ## Usage -Each template can be rendered by specifying the `--directory` flag to cookiecutter. For example, in order to generate a project using the `basic-example-imagespec` template, run: +Each template can be rendered by specifying the `--directory` flag to cookiecutter. For example, in order to generate a project using the `basic-template-imagespec` template, run: - $ cookiecutter https://github.com/flyteorg/flytekit-python-template.git --directory="basic-example-imagespec" + $ cookiecutter https://github.com/flyteorg/flytekit-python-template.git --directory="basic-template-imagespec" This should prompt for a few variables that will be used to set up the project.