01 · RAG and evaluation / Case study
Clinical RAG over synthetic EHR data
A research retrieval-and-answering prototype in which every response stays connected to evidence and a specific patient in a synthetic dataset.
The problem
Build an auditable RAG workflow for NHS-style records without using real patient data.
My contribution
I built the workflow from synthetic record snippets through TF-IDF retrieval and patient filtering to template-based answers with citations. I included a CLI, relevance labels, retrieval and behaviour tests, and documentation of the assumptions.
The correct record comes first
Patient filtering constrains the evidence before an answer is prepared. A similar-sounding passage from another record is not a valid source.
A transparent baseline
TF-IDF makes the behaviour of a simple retriever inspectable. This version uses template-based answers; I do not present it as an open-ended LLM chat system.
Answers need a source
Citations expose the basis of each answer. Missing evidence and requests for diagnosis or treatment have separate, constrained responses.
Evidence and interpretation
What the result actually says.
Reported retrieval results: Precision@3 0.583, Recall@3 1.000 and MRR 1.000 within the project’s defined evaluation.
Recall@3 of 1.000 describes finding the labelled evidence in this synthetic evaluation. Precision@3 of 0.583 reminds us that not every returned passage is relevant. Neither number measures clinical effectiveness.
The dataset is synthetic and the system is an academic prototype, not a clinical product or medical device.
The next useful experiment
Extend the hard-negative set and compare retrievers with identical splits and relevance labels. This is a development direction for the public prototype, separate from the dissertation.
Descriptions checked against the public project documentation on 17 September 2026. Scores are reported project results; the experiments were not rerun for this website update.