Proust RAG: question answering over Du côté de chez Swann
Ask a question about the first volume of Proust's In Search of Lost Time, in French, English or other languages, and get an answer built from quotes of the original French, each cited by chapter and paragraph.
Live demo · proust-rag · proust-pipeline · proust-app
Python · FastAPI · ChromaDB · spaCy · sentence-transformers · Claude Haiku 4.5 · Next.js
What it is
A small retrieval-augmented generation (RAG) service. It finds the sentences in the novel most likely to answer a question, then asks Claude to answer using only those sentences, not its general knowledge of the book.
It's split into three repositories: a data pipeline, a FastAPI service for retrieval and generation, and a Next.js question-answering frontend.
Why I built it
I wanted a project in digital humanities, and I was reading Proust at the time. I love his language and style, and I wanted the tool to respect them: quotes always stay in the original French, and the answer only paraphrases them rather than translating. The project took shape through exploration. I first tried finding all the artworks mentioned in the novel, or all the food, and it gradually settled into a question-answering system grounded in the text. I'm particularly interested in text processing and linguistics, and the natural next direction is collocations: looking for the word combinations that make Proust's style recognizably his.
How it works
- Preparation (offline): spaCy splits the text into sentences, and sentences over the embedding model's 128-token limit are split again at commas. Each chunk is embedded into ChromaDB, and a separate index stores the nouns and names (as lemmas) of each chunk.
- Retrieval: every question gets a semantic search for the five closest chunks. If it also contains a rare noun or name (50 occurrences or fewer), an exact-term search runs too, and the two lists are merged.
- Answer: Claude Haiku 4.5 gets the passages with instructions to answer only from them, quote the French word for word, cite the source, and say so when the text doesn't cover the question.
The madeleine problem
My first version used semantic search only. Searching for "madeleine", the most famous word in the book, returned nothing relevant. The similarity between that one word and the sentence containing it was 0.20, because sentence embeddings capture what a sentence is about, not which words it contains.
The fix was an exact-term layer that's switched on only when the question has a rare term. "Swann" appears in hundreds of chunks and tells you nothing, while "épice" appears in one. The trade-off is that it matches words, not meanings: "madeleine" also finds a Paris church. Full write-up
Evaluation
I test retrieval separately from answer generation, so a missed passage can be told apart from a bad answer. The set has 17 hand-written questions. It's too small to measure overall quality, but it found specific problems. On the 16 questions with an expected answer, retrieval found at least one correct passage for 9. The findings include:
- No "nothing found" signal: a question about Picasso, who isn't in the book, still returned five sentences. The model correctly said the text doesn't mention him, but that depends on the prompt and not on retrieval.
- Spelling: Proust writes "Ver Meer", so a question about "Vermeer" misses those passages.
- One fabricated quote appeared in an answer, which led me to tighten the prompt. One later re-test showed none, and checking every quote against the source text is on my list.
Limitations
Only the first volume is indexed. Exact-term search covers nouns and names in French, one word at a time. Each question is answered independently, with no memory of earlier ones. Language detection is unreliable on short questions.
Next steps
Check every quoted string in an answer against the source text (the fabricated quote), translate questions to French before exact-term search (the cross-language limitation), and grow the evaluation set (17 questions is too small).
Testing and deployment
The API runs in Docker on Hugging Face Spaces and the frontend on Vercel. 16 pytest cases cover routing, lemma extraction and retrieval. Answer generation and the API endpoints have no automated tests yet.