Medical Knowledge RAG
A retrieval-augmented question-answering system over the Merck Manuals so clinicians can get grounded, cited answers instead of paging through hundreds of pages of reference text.
The problem
The Merck Manuals are a comprehensive medical reference, but comprehensive also means slow to search manually under time pressure. The goal was a system that takes a clinical question in plain language and returns an answer that is both accurate and traceable back to the source manual — not a black-box generation.
Approach
Manual sections were split with a RecursiveCharacterTextSplitter into 500–800 character chunks, embedded, and stored in a Chroma vector database. At query time, the question is embedded with the same model and compared against stored vectors with cosine similarity to retrieve the top-k (k = 3–6) most relevant chunks, which are then passed to the language model as grounding context before it answers.
Every answer was manually scored on two axes: groundedness (does the answer actually come from the retrieved context, not the model's own memory?) and relevance (does it actually answer the question asked?).
Sample evaluation results
| Query | Groundedness | Relevance |
|---|---|---|
| Sepsis management in critical care | 4/5 | 5/5 |
| Appendicitis symptoms & surgical treatment | 5/5 | 5/5 |
| Autoimmune hair-loss treatment options | 4-5/5 | 4-5/5 |
| Emergency stroke response protocol | 5/5 | 5/5 |
High groundedness scores across every tested query confirmed the retrieval layer was doing its job: the model's answers tracked the actual retrieved manual text rather than drifting into confident-sounding but unsupported generation — the exact failure mode a medical RAG system can't afford.