Evaluate graph RAG LLM application

[Howie-Arup]

Your Question
Hi, I developed a graph RAG LLM application and want to do evaluation for it like here and here in the documentation. But it seems that the current evaluation is for text document RAG? May I know is there a graph RAG LLM application evaluation feature? My question is actually similar to this guy in stakeoverflow but there is no solution either. Thanks!

[hliuci]

Hi @sahusiddharth Is there any update on this? Or can we do the evaluation just like here with prepared reference graph node data and the retrieved ones?

[jjmachan]

@hliuci @Howie-Arup I'm not sure why we might need a different set of metrics other than the standard ones already present in Ragas?

We have

• Context Precision - Ragas
• Context Recall - Ragas

which should do the job right? all you have to do is convert the nodes to text before passing it into evaluate.

let me know if you need more help here - happy to help 🙂

[hliuci]

Got it @jjmachan, thanks a lot for your help!

[github-actions]

It seems the issue was answered, closing this now.