-
Notifications
You must be signed in to change notification settings - Fork 831
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
495bbd1
commit dc178af
Showing
3 changed files
with
12 additions
and
36 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
# Components Guide | ||
|
||
This guide provides an overview of the different components used inside Ragas. | ||
|
||
## [Prompt Object]() | ||
## [Evaluation Dataset]() | ||
## [Evaluation Sample]() |
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,42 +1,22 @@ | ||
# 📚 Core Concepts | ||
|
||
Ragas aims to create an open standard, providing developers with the tools and techniques to leverage continual learning in their RAG applications. With Ragas, you would be able to | ||
|
||
1. Synthetically generate a diverse test dataset that you can use to evaluate your app. | ||
2. Use LLM-assisted evaluation metrics designed to help you objectively measure the performance of your application. | ||
3. Monitor the quality of your apps in production using smaller, cheaper models that can give actionable insights. For example, the number of hallucinations in the generated answer. | ||
4. Use these insights to iterate and improve your application. | ||
|
||
|
||
## What is RAG and continual learning? | ||
### RAG | ||
|
||
Retrieval augmented generation (RAG) is a paradigm for augmenting LLM with custom data. It generally consists of two stages: | ||
|
||
- indexing stage: preparing a knowledge base, and | ||
|
||
- querying stage: retrieving relevant context from the knowledge to assist the LLM in responding to a question | ||
|
||
### Continual Learning | ||
|
||
Continual learning is concept used in machine learning that aims to learn, iterate and improve ML pipelines over its lifetime using the insights derived from continuous stream of data points. In LLM & RAGs, this can be applied by iterating and improving each components of LLM application from insights derived from production and feedback data. | ||
|
||
<div class="grid cards" markdown> | ||
|
||
- [Evaluation Driven Development](evaluation_driven.md) | ||
- [Components](components/index.md) | ||
|
||
What is EDD? | ||
What are the different components used inside Ragas? | ||
|
||
- [Ragas Metrics](metrics/index.md) | ||
|
||
What metrics are available? How do they work? | ||
|
||
- [Synthetic Test Data Generation](testset_generation.md) | ||
|
||
How to create more datasets to test on? | ||
How to create high quality datasets to test on? | ||
|
||
- [Utilizing User Feedback](feedback.md) | ||
- [Feedback Intelligence](feedback.md) | ||
|
||
How to leverage the signals from user to improve? | ||
How to understand leverage the signals from production data? | ||
|
||
</div> |