Conversation using limited set of answers

[aleksas]

Task:
To make a system/app to respond to questions with a segment or a list of segments from a number of transcribed videos.

Current state:
I've already made a prototype using retrieval example, to process video transcriptions and index them, and play back video segment corresponding to best matched chunk timestamps.

Problem:
The basic retrieval example adaptation lacks "intelligence" of LLM and there is constraint of usable statements from video transcripts.

My idea to try out to feed the best responses to LLM and ask to arrange them to build complete answer. I this a correct path? If so how to properly feed data into it? Are there good and bad ways to d it? Or is there a completely different and better way?