Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Case-insensitive and whitespace handling for context_precision. #289

Conversation

Pauldevillers
Copy link
Contributor

@Pauldevillers Pauldevillers commented Nov 16, 2023

Hello,

Title: Case-insensitive and whitespace handling for response evaluation

Description : This PR modifies the response evaluation in the context_precision.py script. The changes include:

  • Making the response evaluation case-insensitive: The code now converts each response to lowercase before checking if "yes" is in the response. This ensures that the evaluation is not affected by the case of the response.
  • Handling whitespace in responses: The code now strips leading/trailing whitespace from each response before checking if "yes" is in the response. This ensures that the evaluation is not affected by whitespace in the response.

…andle whitespace to identify Yes statement in the output prompt
@Pauldevillers
Copy link
Contributor Author

Seems there is an issue with the 'llama_index.node_parser.simple' package.

@shahules786
Copy link
Member

Hey @Pauldevillers , No worries. There #291 that will fix the issue.

@Pauldevillers
Copy link
Contributor Author

Nice @shahules786 , can we retriger the checks ?

@shahules786
Copy link
Member

closing in respect to #289 merged

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants