Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide TrussUserEnv in API calls and cleanups. #1355

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 1 addition & 10 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -36,14 +36,5 @@ repos:
entry: poetry run mypy
language: python
types: [python]
exclude: ^examples/|^truss/test.+/|model.py$|^truss-chains/.*|^smoketests/.*
pass_filenames: true
- id: mypy
name: mypy-local (3.9)
entry: poetry run mypy
language: python
types: [python]
files: ^truss-chains/.*|^smoketests/.*
args:
- --python-version=3.9
exclude: ^examples/|^truss/test.+/|model.py$
pass_filenames: true
6 changes: 1 addition & 5 deletions docs/examples/09-private-huggingface.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -87,12 +87,8 @@ system_packages: []
```
# Deploying the model

An important note for deploying models with secrets is that
you must use the `--trusted` flag to give the model access to
secrets stored on the remote secrets manager.

```bash
$ truss push --trusted
$ truss push
```

After the model finishes deploying, you can invoke it with:
Expand Down
2 changes: 1 addition & 1 deletion docs/examples/performance/cached-weights.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -139,7 +139,7 @@ You'll need a [Baseten API key](https://app.baseten.co/settings/account/api_keys
We have successfully packaged Llama 2 as a Truss. Let's deploy!

```sh
truss push --trusted
truss push
```

### Step 5: Invoke the model
Expand Down
4 changes: 1 addition & 3 deletions docs/examples/private-model.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -154,10 +154,8 @@ You'll need a [Baseten API key](https://app.baseten.co/settings/account/api_keys

We have successfully packaged a gated model as a Truss. Let's deploy!

Use `--trusted` with `truss push` to give the model server access to secrets stored on the remote host.

```sh
truss push --trusted
truss push
```

Wait for the model to finish deployment before invoking.
Expand Down
1,997 changes: 956 additions & 1,041 deletions poetry.lock

Large diffs are not rendered by default.

8 changes: 4 additions & 4 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -69,8 +69,7 @@ truss = "truss.cli.cli:truss_cli"
# isolated.
[tool.poetry.dependencies]
# "base" dependencies.
# "When using chains, 3.9 will be required at runtime, but other truss functionality works with 3.8.
python = ">=3.8,<3.13"
python = ">=3.9,<3.13"
huggingface_hub = ">=0.25.0"
pydantic = ">=1.10.0" # We cannot upgrade to v2, due to customer constraints.
PyYAML = ">=6.0"
Expand Down Expand Up @@ -105,6 +104,7 @@ tenacity = { version = "^8.0.1", optional = false }
watchfiles = { version = "^0.19.0", optional = false }



[tool.dependency_metadata]
# `base` / `main` deps which are non-optional are always included and don't need to be added here.
Jinja2 = { components = "other" }
Expand Down Expand Up @@ -176,7 +176,7 @@ requires = ["poetry-core>=1.2.1"]

[tool.mypy]
ignore_missing_imports = true
python_version = "3.8"
python_version = "3.9"
plugins = ["pydantic.mypy"]

[tool.pytest.ini_options]
Expand All @@ -188,7 +188,7 @@ addopts = "--ignore=smoketests"

[tool.ruff]
src = ["truss", "truss-chains", "truss-utils"]
target-version = "py38"
target-version = "py39"
line-length = 88
lint.extend-select = [
"I", # isort
Expand Down
8 changes: 0 additions & 8 deletions truss-chains/truss_chains/__init__.py
Original file line number Diff line number Diff line change
@@ -1,11 +1,3 @@
import sys

if (sys.version_info.major, sys.version_info.minor) <= (3, 8):
raise RuntimeError(
"Python version 3.8 or older is not supported for `Truss-Chains`. Please"
"upgrade to Python 3.9 or newer. You can still use other Truss functionality."
)
del sys
import pydantic

pydantic_major_version = int(pydantic.VERSION.split(".")[0])
Expand Down
10 changes: 6 additions & 4 deletions truss-chains/truss_chains/deployment/deployment_client.py
Original file line number Diff line number Diff line change
Expand Up @@ -569,8 +569,9 @@ def __init__(

def _patch(self) -> None:
exception_raised = None
with log_utils.LogInterceptor() as log_interceptor, self._console.status(
" Live Patching Model.\n", spinner="arrow3"
with (
log_utils.LogInterceptor() as log_interceptor,
self._console.status(" Live Patching Model.\n", spinner="arrow3"),
):
try:
gen_truss_path = code_gen.gen_truss_model_from_source(self._source)
Expand Down Expand Up @@ -729,8 +730,9 @@ def _code_gen_and_patch_thread(
def _patch(self, executor: concurrent.futures.Executor) -> None:
exception_raised = None
stack_trace = ""
with log_utils.LogInterceptor() as log_interceptor, self._console.status(
" Live Patching Chain.\n", spinner="arrow3"
with (
log_utils.LogInterceptor() as log_interceptor,
self._console.status(" Live Patching Chain.\n", spinner="arrow3"),
):
# Handle import errors gracefully (e.g. if user saved file, but there
# are syntax errors, undefined symbols etc.).
Expand Down
21 changes: 0 additions & 21 deletions truss-utils/README.md

This file was deleted.

75 changes: 0 additions & 75 deletions truss-utils/poetry.lock

This file was deleted.

17 changes: 0 additions & 17 deletions truss-utils/pyproject.toml

This file was deleted.

1 change: 0 additions & 1 deletion truss-utils/truss_utils/__init__.py

This file was deleted.

17 changes: 0 additions & 17 deletions truss-utils/truss_utils/image.py

This file was deleted.

32 changes: 0 additions & 32 deletions truss-utils/truss_utils/s3.py

This file was deleted.

4 changes: 2 additions & 2 deletions truss/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@
__version__ = get_version(__name__, Path(__file__).parent.parent)


def version():
return __version__
def version() -> str:
return __version__ or ""


from truss.api import login, push, whoami
Expand Down
12 changes: 9 additions & 3 deletions truss/api/__init__.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import warnings
from typing import TYPE_CHECKING, Optional, Type, cast

if TYPE_CHECKING:
Expand Down Expand Up @@ -57,7 +58,7 @@ def push(
publish: bool = False,
promote: bool = False,
preserve_previous_production_deployment: bool = False,
trusted: bool = False,
trusted: Optional[bool] = None,
deployment_name: Optional[str] = None,
environment: Optional[str] = None,
progress_bar: Optional[Type["progress.Progress"]] = None,
Expand All @@ -76,7 +77,7 @@ def push(
preserve_previous_production_deployment: Preserve the previous production deployment’s autoscaling
setting. When not specified, the previous production deployment will be updated to allow it to
scale to zero. Can only be use in combination with `promote` option.
trusted: Give Truss access to secrets on remote host.
trusted: [DEPRECATED]
deployment_name: Name of the deployment created by the push. Can only be
used in combination with `publish` or `promote`. Deployment name must
only contain alphanumeric, ’.’, ’-’ or ’_’ characters.
Expand All @@ -86,6 +87,12 @@ def push(
Returns:
The newly created ModelDeployment.
"""
if trusted is not None:
warnings.warn(
"`trusted` is deprecated and will be ignored, all models are "
"trusted by default now.",
DeprecationWarning,
)

if not remote:
available_remotes = RemoteFactory.get_available_config_names()
Expand All @@ -112,7 +119,6 @@ def push(
tr,
model_name=model_name,
publish=publish,
trusted=trusted,
promote=promote,
preserve_previous_prod_deployment=preserve_previous_production_deployment,
deployment_name=deployment_name,
Expand Down
5 changes: 2 additions & 3 deletions truss/base/truss_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -859,16 +859,15 @@ def obj_to_dict(obj, verbose: bool = False):
return d


# TODO(marius): consolidate this with config/validation:
def _infer_python_version() -> str:
return f"py{sys.version_info.major}{sys.version_info.minor}"


def map_local_to_supported_python_version() -> str:
return map_to_supported_python_version(_infer_python_version())
return _map_to_supported_python_version(_infer_python_version())


def map_to_supported_python_version(python_version: str) -> str:
def _map_to_supported_python_version(python_version: str) -> str:
"""Map python version to truss supported python version.

Currently, it maps any versions greater than 3.11 to 3.11.
Expand Down
8 changes: 4 additions & 4 deletions truss/cli/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -658,9 +658,10 @@ def push_chain(
num_failed = 0
# Logging inferences with live display (even when using richHandler)
# -> capture logs and print later.
with LogInterceptor() as log_interceptor, rich.live.Live(
table, console=console, refresh_per_second=4
) as live:
with (
LogInterceptor() as log_interceptor,
rich.live.Live(table, console=console, refresh_per_second=4) as live,
):
while True:
table, statuses = _create_chains_table(service)
live.update(table)
Expand Down Expand Up @@ -1198,7 +1199,6 @@ def push(
tr,
model_name=model_name,
publish=publish,
trusted=True,
promote=promote,
preserve_previous_prod_deployment=preserve_previous_production_deployment,
deployment_name=deployment_name,
Expand Down
Loading
Loading