Medical RAG Chatbot

The interesting problem is refusal

Retrieval is the easy part. Chunk a corpus, embed it, search it, hand the top results to a model — that is a weekend, and every RAG tutorial ends there. The part that decides whether the thing is usable is what happens when the corpus does not contain the answer.

A model handed thin or irrelevant context will still answer. It will answer fluently, in the register of the documents it did see, and a reader has no way to tell that from a grounded answer. In a medical context that is not a quality problem, it is the whole risk.

So the work here went into getting it to say "my sources do not cover that" — which is a grounding and instruction problem, not a retrieval one.

How it is put together

A PDF corpus is chunked and embedded into Pinecone, retrieved semantically at query time, and served through Flask with Llama 3 doing the generation. LangChain wires the retrieval and prompting together.

The prompt constrains the model to the retrieved context and instructs it to decline rather than extrapolate. Retrieval results below a relevance threshold are treated as no context at all, rather than being passed through as weak evidence.

What I am building next

Grounding here works at the prompt and threshold level. The next step is a validation pass that checks the generated answer back against the retrieved chunks before it reaches the user — moving from removing the opportunity to hallucinate to structurally preventing it.