Skip to content

Commit

Permalink
update docs
Browse files Browse the repository at this point in the history
  • Loading branch information
akshayballal95 committed Jan 9, 2025
1 parent 47c6542 commit 09e6a5e
Show file tree
Hide file tree
Showing 2 changed files with 46 additions and 2 deletions.
27 changes: 27 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -282,6 +282,33 @@ print(data[0].metadata)

```

### Using ONNX Models

To use ONNX models, you can either use the `ONNXModel` enum or the `model_id` from the Hugging Face model.

```python
model = EmbeddingModel.from_pretrained_onnx(
WhichModel.Bert, model_name = ONNXModel.AllMiniLML6V2Q
)
```

For some models, you can also specify the dtype to use for the model.

```python
model = EmbeddingModel.from_pretrained_onnx(
WhichModel.Bert, ONNXModel.ModernBERTBase, dtype = Dtype.Q4F16
)
```

Using the above method is best to ensure that the model works correctly as these models are tested. But if you want to use other models, like finetuned models, you can use the `hf_model_id` and `path_in_repo` to load the model like below.

```python
model = EmbeddingModel.from_pretrained_onnx(
WhichModel.Jina, hf_model_id = "jinaai/jina-embeddings-v2-small-en", path_in_repo="model.onnx"
)
```
To see all the ONNX models supported with model_name, see [here](../guides/onnx_models)




Expand Down
21 changes: 19 additions & 2 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -239,11 +239,28 @@ The output is a list of documents with their relevance scores and rank for each

### Using ONNX Models

To use ONNX models, you can either use the `ONNXModel` enum or the `model_id` from the Hugging Face model.

```python

model = EmbeddingModel.from_pretrained_onnx(
WhichModel.Bert, model_name = ONNXModel.AllMiniLML6V2Q
)
```
For some models, you can also specify the dtype to use for the model.

```python
model = EmbeddingModel.from_pretrained_onnx(
WhichModel.Bert, ONNXModel.ModernBERTBase, dtype = Dtype.Q4F16
)
```

Using the above method is best to ensure that the model works correctly as these models are tested. But if you want to use other models, like finetuned models, you can use the `hf_model_id` and `path_in_repo` to load the model like below.

```python
model = EmbeddingModel.from_pretrained_onnx(
WhichModel.Bert, ONNXModel.AllMiniLML6V2Q
WhichModel.Jina, hf_model_id = "jinaai/jina-embeddings-v2-small-en", path_in_repo="model.onnx"
)
```
To see all the ONNX models supported, see [here](../guides/onnx_models)
To see all the ONNX models supported with model_name, see [here](../guides/onnx_models)

0 comments on commit 09e6a5e

Please sign in to comment.