P74Generative AI / RAGAdvancedDesign18 min
Design an Evaluation Set for a RAG System
Design RAG evaluation that separates retrieval from grounding.
#design#rag#evaluation
Scenario
An HR policy assistant is tested only with five easy developer-written FAQs.
Problem statement
Design a credible evaluation set.
Your Task
- Include varied questions.
- Separate retrieval and answer checks.
- Include unsupported cases.
Related Concepts
GroundednessRetrieval
Ready to count this problem?
Mark it complete when you have reasoned through the solution in your own words.