Docsite search: Test performance for integration with the AI Assistant

[hanna-paasivirta]

Once the docsite search RAG is done, test its performance to see how it could be leveraged in the AI Assistant.

Decide on the testing scenarios. The simplest is doing a search once per conversation to add relevant documentation based on adaptors in use etc. to the prompt.

Manually test cases, then generate more test cases. For each testing scenario, evaluate:

• whether the docsite RAG improves the quality of answers
• time & prompt & embeddings api cost