Vector search — meaning-based, at scale

01Keyword search vs semantic search — feel it

Chapter 8 named the problem; here you watch it. The same query, searched two ways over the same notices — one matching letters, one matching meaning:

Keyword search is not useless — it is exact, fast, and unbeatable when you know the precise word (a product code, a name). Semantic search wins when the user’s words and the document’s words differ but the meaning matches — which, for questions asked in human language, is most of the time. Real systems often use both together (called hybrid search), but the new and powerful half is the semantic one, so that is what we build.

You use both every week without naming them. On IRCTC you type “12951” and expect that exact train — that is keyword search, and anything cleverer would be a bug. On Swiggy you type “something light, not too spicy” and expect idli and curd rice to come back, even though not one of those words appears on the dish. That second one is semantic search, and it is what your notice-board project needs: a student will ask “when do I have to pay?” about a notice that only ever says fee deadline.

02First, chunking — cut documents into bites

Before you can search by meaning, one preparation step almost every tutorial fumbles. You do not embed a whole 40-page document as one position — that would blur every topic in it into a single muddy average, useless for finding a specific answer. Instead you cut the document into smaller pieces first. That cutting is called chunking, and each piece gets its own embedding and its own dot on the map.

The judgment is in the size. Chunks too big mix several ideas and retrieve fuzzily. Chunks too small lose the context that made them meaningful — a sentence ripped from its paragraph. The honest sweet spot is a paragraph-ish size, split at natural boundaries (paragraphs, sections) rather than blindly every N characters. And a professional trick: overlap the chunks slightly — let each one repeat the last sentence or two of the previous — so an idea that straddles a boundary is not sliced in half and lost. Getting chunking right quietly decides whether your whole search feels smart or dumb; it is the unglamorous step that matters most.

03Why your Chapter 4 database can’t do this

You already know a database (Chapter 4) that stores rows and finds them fast. Why not use it? Because it finds things by exact match and sorting — where email = ‘x’, or all rows ordered by date. It has no notion of “find the rows whose meaning is nearest to this position in a 1,500-dimensional space”. Ask a normal database that and it would have to compare your query against every single row, one by one — fine for a hundred, hopeless for ten million.

04The vector database

So a specialised kind of database exists for exactly this job: the vector database. Its one talent is storing millions of embeddings and answering, near-instantly, “here are the k positions closest to this one” — a nearest-neighbour search. Names you will meet: Pinecone (hosted, easy to start), Qdrant and Chroma(open-source, run them yourself), and pgvector — an add-on that gives the Postgres you already know from Chapter 4 this exact superpower, so you often do not even need a separate database.

The trick that makes it fast is worth one honest sentence: a vector database does not truly check every point either. It builds a smart index ahead of time (the Chapter 4 idea, evolved) that lets it leap to the right neighbourhood and check only the points nearby. To get that speed it accepts being approximatelyright — occasionally missing the very closest point for one just slightly further. For search, a near-perfect answer in one millisecond beats a perfect answer in ten seconds every time.

05The two phases — indexing and querying

Every semantic-search system, including the ones you will build, has two clearly separate phases. Keep them apart in your head and the whole thing stays simple.

Indexing (done ahead of time, once): take your documents, chunk them, embed every chunk with your chosen model (Chapter 8), and store each embedding — plus the original text it came from — in the vector database. This is a batch job you run when documents are added or change, not while a user waits.

Querying (done live, per search): take the user’s query, embed it with the same model, ask the vector database for the nearest chunks, and return them. Embed one query, one fast nearest-neighbour lookup, done — the reason search feels instant is that all the heavy work already happened during indexing.

06Store the text with the vector

A beginner mistake worth stopping on: people store the embedding and forget to store the original text beside it. But an embedding is a position — a list of numbers — and you cannot show a user a list of numbers. So each entry in the vector database keeps the vector (for finding) and the source text and a little metadata like which document and page it came from (for showing and citing). That stored text is what the next chapter feeds to an LLM, and that stored source is how a RAG answer can point at where it came from. Design for it now.

07How this fits the workflow

Nothing here breaks the system you built in Chapters 2–4; it adds one box to the drawing, exactly as Chapter 1 promised growth would. Your backend gains an indexing job and a search endpoint; the vector database sits alongside your normal one. With Claude Code, wiring a hosted vector database or pgvector is a well-trodden path — the judgment you bring is the design: how you chunk, which embedding model, how many neighbours to fetch, and what metadata to keep. Put those decisions in your docs/ before building, the same discipline as every chapter.

08Do this today

Build real semantic search over your notice board. Chunk each notice (they are short — one chunk each is fine to start), embed them, and store the embeddings with their text in pgvector or a free hosted vector database. Then a search box: embed the query, fetch the three nearest notices, show them. Now search “when do I pay” and watch it surface the notice about tuition dates that shares not one word with your query. That moment — a search that understands instead of matches — is the payoff of two chapters, and the exact machine the next chapter turns into a system that answers.