Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Same metrics function with different content must be rewrite the function ?? #1715

Closed
YYH211 opened this issue Nov 28, 2024 · 2 comments
Closed
Labels
bug Something isn't working module-metrics this is part of metrics module

Comments

@YYH211
Copy link

YYH211 commented Nov 28, 2024

Now I want to apply one of the metrics functions in the evaluation metrics to different content.
For example, the SemanticSimilarity function by default takes {"reference", "response"} to measure the similarity between two pieces of content. Now I want to measure the similarity between {"user_input", "retrieved_contexts"}.
How should I proceed? Does it mean that I have to rewrite the whole method myself?

@YYH211 YYH211 added the bug Something isn't working label Nov 28, 2024
@dosubot dosubot bot added the module-metrics this is part of metrics module label Nov 28, 2024
@jjmachan
Copy link
Member

hey there @YYH211 sadly this is not possible in metrics other than general purpose metrics sadly https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/general_purpose/

but I can make this fix for you today and merge it in 🙂

@sahusiddharth sahusiddharth added the waiting 🤖 waiting for response. In none will close this automatically label Jan 11, 2025
Copy link

Closing after 8 days of waiting for the additional info requested.

@github-actions github-actions bot removed the waiting 🤖 waiting for response. In none will close this automatically label Jan 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working module-metrics this is part of metrics module
Projects
None yet
Development

No branches or pull requests

3 participants