[Feature][Executor] Refine the image startup scripts to support customized executor image #2762

PeiwenGaoMS · 2024-04-11T10:56:39Z

Description

Background

To facilitate customers in choosing their desired Python version within the executor image (default version: 3.9), we will offer a simple and efficient solution that allows users to quickly build custom images.

Implementation

Integrate an entry point into promptflow-core.
Include uvicorn and gunicorn as extra dependencies in promptflow-core.
Implement run mode (serving/compute) detection within the entry point to start the app accordingly.

Usage

Once the changes from this PR are released to the public version of promptflow, customers can easily build their desired Python version of the executor image using the Dockerfile below, and then add it to the flow yaml to create an automatic runtime for use.

# Choose different python base image
FROM python:3.11

# Install promptflow related package
RUN pip install promptflow_tracing
RUN pip install promptflow_core[executor-service,azureml-serving]
RUN pip install promptflow_devkit
RUN pip install promptflow

# Set port
USER root:root
ENV PORT 8000
EXPOSE $PORT

CMD ["pfapp"]

# Add this part to flow yaml
environment:
  image: <image-name>

Test Links

PR Changes Summary

This pull request primarily updates the src/promptflow-core package to include a new execution_app.py module and modifies the pyproject.toml file to include new dependencies and scripts. The execution_app.py module includes a new RunMode class, functions to get the number of processes based on the CPU core number, check if a given model directory contains certain files, get the run mode, and start the server based on the run mode. The pyproject.toml file has been updated to include uvicorn and gunicorn as dependencies and a new script to start the server.

Here are the most important changes:

src/promptflow-core/promptflow/execution_app.py: A new module that includes a RunMode class, functions to get the process number based on the CPU core number, check if a given model directory contains certain files, get the run mode, and start the server based on the run mode.

Updates to pyproject.toml:

src/promptflow-core/pyproject.toml: Added uvicorn and gunicorn as dependencies for the executor service and AzureML serving, respectively.
src/promptflow-core/pyproject.toml: Added a new script, pfapp, to start the server.

All Promptflow Contribution checklist:

The pull request does not introduce [breaking changes].
CHANGELOG is updated for new features, bug fixes or other significant changes.
I have read the contribution guidelines.
Create an issue and link to the pull request to get dedicated review from promptflow team. Learn more: suggested workflow.

General Guidelines and Best Practices

Title of the pull request is clear and informative.
There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

Pull request includes test coverage for the included changes.

github-actions · 2024-04-11T10:58:36Z