How does Giskard generate with the reference_answer?

[DinaberryPi]

Hi,

In the documentation, it states that "RAGET can generate automatically a list of question, reference_answer and reference_context from the knowledge base of the RAG." I wonder how it generate the 'reference_answer'. Can someone explain what happens under the hood? How do I know if I can trust the reference_answer align with the ground truth?

[alexcombessie]

Hey @pierlj - When you have some time, would you be able to answer @DinaberryPi 🙏 ? Many thanks!

[pierlj]

Hi @DinaberryPi , the reference_answer is generated at the same time as the question. What happen is that we select a piece of context inside the knowledge base and pass it to the LLM, which is prompted to generate a question and its answer. You can consider the reference_answer as the ground truth for the generated question.