Personal project · 2026 · Engineering

Meeting Notes RAG

A retrieval-augmented generation pipeline that answers questions about my meetings. Wispr Flow transcribes every call I take and writes the transcript into a Notion database. This system pulls those pages from Notion, embeds them into a local vector store, and answers questions from a terminal chat loop using only the excerpts it retrieved. 118 meetings, one file, no infrastructure.

RAGLLM ApplicationsPythonInformation RetrievalDeveloper Tooling

Overview

ProjectPersonal

RoleEngineering

Date2026

StackPython 3.10+, LangChain (LCEL), Notion API, OpenAI, Qdrant, Pydantic, pytest, uv

Repogithub.com/matissepage/rag-system →

The problem.

Notion's search matches keywords. It finds the meeting where someone typed the exact phrase and misses the one where the same decision was reached in different words. Questions like “what did we agree on with Qobuz in the last workshop?” are not keyword questions. They are questions about what a conversation concluded.

The deeper problem was what I was actually searching. These are transcripts, not notes. A written note is already a summary: someone decided what mattered and wrote that down. A transcript is forty minutes of unedited speech. It contains filler, false starts, three unrelated topics in one document, and the one sentence that records a real decision buried mid-paragraph in language that never restates what the meeting was about.

That distinction shaped most of what follows. Standard RAG tutorials assume documents that are about something. A meeting transcript is about four things, none of them announced.

The goal

Build something I would use daily. That meant three things: answers grounded strictly in my transcripts, an honest “I don't know” when they don't cover it, and an index that keeps itself current as Wispr Flow adds meetings.

Pipeline

The pipeline, end to end.

1 · CaptureWispr Flow sits in the meeting and transcribes it.

2 · StoreThe transcript lands in my Notion meetings database as a page, with a title, a date, a URL and tags as properties, and the transcript itself as the body.

3 · IndexPulls those pages through the Notion API, splits each transcript into chunks, embeds them, and writes them to Qdrant on disk.

4 · AskA terminal loop takes a question, decides how to retrieve for it, and answers strictly from what it retrieved.

I wrote steps 3 and 4. Steps 1 and 2 already existed and cost me nothing per meeting, which is exactly why three years of transcripts had accumulated without me ever reading them back.

Approach

Approach.

Retrieve wide, then let a model narrowThe naive pipeline retrieves five chunks and hands all five to the answering model. Vector similarity is good at recall and mediocre at precision, and transcripts make that worse: verbatim speech shares generic vocabulary with every other conversation, so the top five almost always contained two useful excerpts and three that merely sounded like meetings.So retrieval and selection became separate steps. Qdrant returns the 15 nearest chunks, casting a deliberately wide net, and gpt-4o-mini scores each one from 0 to 10 for actual relevance. Scores come back through structured output as validated Pydantic objects rather than parsed text. Anything below a floor of 3 is dropped, and at most 5 survive.The important part is that the reranker is allowed to return nothing. Dropping weak matches rather than always filling five slots is what lets a question my meetings don't cover reach the answering model with an empty context, so it says “there is no information about this” instead of confabulating from three irrelevant excerpts.

Read the question's time scope before retrieving anything“What did we decide last month?” has a date in it that embeddings cannot see. A chunk from March and a chunk from September are equally close to that question in vector space.Before retrieval, gpt-4o-mini reads the question and returns a TimeScope: an optional start and end date, a prefer_recent flag, and an aggregate mode. Today's date is injected into the prompt so relative phrasing resolves. When a period is named, a Qdrant range filter restricts search to it. When prefer_recent is set, a chunk's position in the candidate set's date span adds up to 2 points to its rerank score, so recency shifts the ranking without overriding a large difference in topical fit.The failure mode here is a misread date range, which would return nothing and read as a confident “there is no information about this”. So if the filter matches zero chunks, the search retries unfiltered. A bad date guess degrades to unranked results rather than to a false negative.

Stamp every chunk with its own provenanceEach 1500-character chunk carries a Meeting / Date header, not just the first chunk of a page. This costs a little context budget and buys two things: the answering model can tell excerpts apart by date and cite which meeting a claim came from, and the title lands in the embedded text of every chunk.That second effect matters far more for transcripts than for written documents. A chunk drawn from the middle of a call is a stretch of dialogue that may never name the client, the project or the decision under discussion. Without the header it is close to unretrievable. With it, the subject of the meeting travels with every fragment of it.The answering prompt is written for the same reality. It instructs the model to summarize what the context says “even where it is terse, fragmentary, or written in another language”, because transcribed speech is all three, and an earlier version of the prompt treated messy excerpts as unusable and declined to answer from them.

Sync incrementally, and refuse to answer from a stale indexEvery meeting carries a content_hash and a source_id taken from its Notion page ID. On startup the system pulls the current state of the database and compares hashes against what is stored: new meetings are embedded, edited pages are replaced, deleted ones are removed. A new meeting costs one transcript's worth of embeddings, not a full rebuild, which matters when the capture layer is adding pages on its own schedule.It also writes the settings that shaped the index (embedding model, chunk size, overlap, schema version) to index_meta.json, and checks them on startup. If I change the embedding model, the system refuses to answer rather than quietly returning results from mismatched vectors. A wrong answer that looks right is worse than no answer.

Failure case

The case that broke it.

Question“What was my latest meeting about?”AnswerThere is no information about this in the meeting notes.

That was obviously wrong. There is always a newest meeting. I spent a while treating it as a tuning problem, adjusting k and the rerank floor, before recognizing it as a hard limit of the approach.

That question is ORDER BY date DESC LIMIT 1 over metadata. Embeddings cannot express it. Retrieval latched onto the literal word “meeting” and returned chunks from pages that Wispr Flow had auto-titled “Untitled meeting”, while the genuinely newest transcript was never in the candidate set at all. Reranking could not rescue it, because reranking only reorders what retrieval already found.

So these questions skip retrieval entirely. When the scope analyzer sets an aggregate mode, a separate path scrolls page metadata, deduplicates chunks back into meetings, sorts by timestamp, and builds the context directly.

ModeQuestionContext handed to the model

Modenewest / oldest

Question“What was my latest meeting about?”

Context handed to the modelA header naming the meeting, plus that transcript's chunks

Modelist

Question“what meetings did I have in April 2026?”

Context handed to the modelA dated list of every meeting in range

Modecount

Question“how many meetings in May 2025?”

Context handed to the modelThe count and the span it covers

Two details made this work. The computed context is labelled as a calendar summary computed directly from the index, so the model reads it as fact rather than mistaking it for a transcript excerpt and refusing to answer from it. And meetings are keyed on the Notion page ID rather than the title, because auto-captured titles repeat heavily: 16 of my 118 meetings are titled “BCE Sync”, another 5 are “BCE SYNC”, and two share both a title and a date. Keying on title would have merged them into one meeting.

The classification rides along on the scope call that already runs for every question, so it costs no extra latency.

I also drew a boundary and left it there. “How many meetings did we have with Holovox” combines an aggregate with an entity, and “Holovox” has to be matched against transcript content, which is a semantic problem again. Answering that properly needs hybrid search. The metadata path handles dates only, and I documented that it does.

Outcome

Outcome.

✓Answers that cite their source. Every answer names the meeting and date it drew from, because every chunk carries that header into the context.

✓A real “I don't know”. The reranker's ability to return nothing, plus a prompt that instructs the model to say so, means an uncovered question gets a refusal instead of a plausible invention.

✓A calendar the transcripts didn't have. Aggregate questions that vector search structurally cannot answer are now answered exactly, from metadata, in one pass.

✓Three years of transcripts became queryable. The capture layer was already running and already free. The gap was between having the data and being able to ask it anything.

✓49 tests, no API key required. Frontmatter parsing, chunk headers, date filters, rerank scoring, recency weighting, sync behaviour, and every calendar answer are covered against a fake Qdrant client. The suite runs offline in under a second, which is what made the reranking and recency logic safe to keep changing.

✓Under 900 lines, zero services. One main.py, a local Qdrant file, and an OpenAI key. Nothing to deploy, nothing to pay for while idle.

Reflection

Reflection.

The interesting work in RAG was not the retrieval. It was noticing which questions retrieval is structurally wrong for, and building a different path for those instead of tuning parameters until the symptom moved.

Most of the rest came from taking the input seriously. Building over transcripts rather than notes is a different problem than the tutorials describe, and nearly every decision I would defend in this project traces back to that: retrieving wide because speech has poor precision, stamping the title on every chunk because dialogue rarely names its own subject, and writing a prompt that expects fragments rather than prose.

The recurring theme was making the system's uncertainty legible rather than hiding it. A reranker that can return nothing. A stale index that refuses to answer. A date filter that falls back instead of failing closed. Each of those is a place where the easy implementation produces a confident wrong answer and the honest one produces a visible gap. For a tool answering questions about my own work, the visible gap is worth more.

Known limits

No conversation memory, so follow-ups don't carry context

Three LLM calls per question, so it's slower and pricier than plain vector retrieval

The scope analyzer is its own failure surface, prone to setting prefer_recent on questions that merely mention a period

Those are the next things to fix, and hybrid search is what unlocks most of them.

All Projects