Skip to content

Commit

Permalink
Remove merlin sdk from user's conda.yaml (#495)
Browse files Browse the repository at this point in the history
<!--  Thanks for sending a pull request!  Here are some tips for you:

1. Run unit tests and ensure that they are passing
2. If your change introduces any API changes, make sure to update the
e2e tests
3. Make sure documentation is updated for your PR!

-->

**What this PR does / why we need it**:
<!-- Explain here the context and why you're making the change. What is
the problem you're trying to solve. --->

After releasing Merlin 0.36.0-rc3 that include pyfunc_v3 to support
model observability features, users that have merlin-sdk as explicit
dependencies in their conda.yaml will get following error:

```
ImportError: cannot import name 'PyFuncOutput' from 'merlin.pyfunc' ...
```

This error is caused because the pyfunc and batch prediction job base
image is installing local version of merlin-sdk (tag v0.36.0-rc3) that
have PyFuncOutput and other classes. However, during image building,
conda will install latest stable version of Merlin (namely 0.35.3) which
will override the local version of merlin-sdk and raise that error.

This PR removes merlin-sdk in user's conda.yaml, from this:

```
name: merlin-model
dependencies:
  - python=3.8
  - pip=22.2.2
  - pip:
      - mlflow
      - merlin-sdk
      - merlin-sdk==0.35.0
      - merlin-sdk<0.35.0
      - merlin-sdk>0.35.0
      - merlin-sdk<=0.35.0
      - merlin-sdk>=0.35.0
      - merlin-sdk>0.35.0,<0.36.0
      - merlin-sdk>=0.35.0,<0.36.0
      - merlin-sdk>0.35.0,<=0.36.0
      - merlin-sdk>=0.35.0,<=0.36.0
```

to this:

```
name: merlin-model
dependencies:
  - python=3.8
  - pip=22.2.2
  - pip:
      - mlflow==1.26.1
```

**Which issue(s) this PR fixes**:
<!--
*Automatically closes linked issue when PR is merged.
Usage: `Fixes #<issue number>`, or `Fixes (paste link of issue)`.
-->

Fixes error image building if user specified merlin-sdk without version
in their conda.yaml

**Does this PR introduce a user-facing change?**:
<!--
If no, just write "NONE" in the release-note block below.
If yes, a release note is required. Enter your extended release note in
the block below.
If the PR requires additional action from users switching to the new
release, include the string "action required".

For more information about release notes, see kubernetes' guide here:
http://git.k8s.io/community/contributors/guide/release-notes.md
-->

```release-note

```

**Checklist**

- [ ] Added unit test, integration, and/or e2e tests
- [x] Tested locally
- [ ] Updated documentation
- [ ] Update Swagger spec if the PR introduce API changes
- [ ] Regenerated Golang and Python client if the PR introduce API
changes
  • Loading branch information
ariefrahmansyah authored Nov 27, 2023
1 parent d693877 commit 3340e2f
Show file tree
Hide file tree
Showing 2 changed files with 2 additions and 0 deletions.
1 change: 1 addition & 0 deletions python/batch-predictor/docker/app.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -25,5 +25,6 @@ RUN if [[ ! -z "$GOOGLE_APPLICATION_CREDENTIALS" ]]; then gcloud auth activate-s
RUN gsutil -m cp -r ${MODEL_URL} .
RUN /bin/bash -c ". activate ${CONDA_ENVIRONMENT} && \
sed -i 's/mlflow[><=]\{0,1\}.*$/mlflow==1.26.1/g' ${HOME}/model/conda.yaml && \
sed -i '/merlin-sdk/d' ${HOME}/model/conda.yaml && \
conda env update --name ${CONDA_ENVIRONMENT} --file ${HOME}/model/conda.yaml && \
python ${HOME}/merlin-spark-app/main.py --dry-run-model ${HOME}/model"
1 change: 1 addition & 0 deletions python/pyfunc-server/docker/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ RUN if [ ! -z "$GOOGLE_APPLICATION_CREDENTIALS" ]; then gcloud auth activate-ser
RUN gsutil cp -r ${MODEL_URL} .
RUN /bin/bash -c ". activate merlin-model && \
sed -i 's/mlflow[><=]\{0,1\}.*$/mlflow==1.26.1/g' model/conda.yaml && \
sed -i '/merlin-sdk/d' model/conda.yaml && \
conda env update --name merlin-model --file model/conda.yaml && \
python -m pyfuncserver --model_dir model --dry_run"

Expand Down

0 comments on commit 3340e2f

Please sign in to comment.