-
Notifications
You must be signed in to change notification settings - Fork 1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: Updating retrieve online documents v2 to work for other fields for sq… #5082
Conversation
…lite Signed-off-by: Francisco Javier Arceo <[email protected]>
Signed-off-by: Francisco Javier Arceo <[email protected]>
Signed-off-by: Francisco Javier Arceo <[email protected]>
Signed-off-by: Francisco Javier Arceo <[email protected]>
2623df8
to
0dff5f1
Compare
@@ -196,6 +196,17 @@ def __str__(self): | |||
UnixTimestamp: pyarrow.timestamp("us", tz=_utc_now().tzname()), | |||
} | |||
|
|||
FEAST_VECTOR_TYPES: List[Union[ValueType, PrimitiveFeastType, ComplexFeastType]] = [ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
wonder if this is used somewhere? :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
vector_bin = serialize_f32( | ||
val.float_list_val.val, config.online_store.vector_len | ||
) # type: ignore | ||
if feature_type_dict[feature_name] in FEAST_VECTOR_TYPES: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@HaoXuAI see here!
Technically there's a flaw here I need to resolve because you have to pass in a query embedding for just pure text search, which is silly. |
What this PR does / why we need it:
This PR enables full text search for the
retrieve_online_documents/
endpoint for SQLite Vec. It also establishes a new parameter in the SDK method calledquery_string
that can be passed to use key word search. There are a number of limitations with this approach as thetop_k
parameter can be misleading (as evident by the example). This offers a good start for keyword search that leverages the existing vector retrieval endpoint. As a next step, enabling hybrid search would be beneficial.It makes keyword search as simple as:
feature_server.py
:feature_store.py
:feature_view.py
:milvus.py
:online_store.py
:sqlite.py
:passthrough_provider.py
andprovider.py
:types.py
:example_feature_repo_1.py
:Which issue(s) this PR fixes:
#5081
#5073
Misc