Query-time retrieval: load the vector store, embed the question, and tune threshold and top-k to return the right chunks.
This second basic example focuses on the retrieval half of the pipeline. The vector store already exists, so the system now loads that persistent store and uses it to pull back the most relevant chunks for a user's question.
The most important implementation rule: the embedding model used for the user question must match the embedding model used during ingestion. If stored chunks were embedded with one model and the incoming query is embedded with another, the similarity scores become unreliable because the vectors are no longer comparable in the same space.
The retriever is then configured with two important controls:
Why tuning matters: if the threshold is too low, the generator receives noisy context. If it is too high, the retriever may return no chunks at all, even when the answer is present. This example teaches that retrieval quality is a balancing problem, not a fixed default.
Source-backed reinforcement: these points add detail beyond short-duration UI hints and emphasize production tradeoffs.
First-time learner note: Build deterministic baseline chains first (prompt -> model -> parser), then add retrieval, memory, or tools only when the baseline is stable.
Production note: Keep contracts explicit at each boundary: input variables, output schema, retries, and logs. This is what keeps orchestration reliable at scale.
This second example is the query-time half of RAG. The database already exists, so the problem shifts from 'how do we store knowledge?' to 'how do we pull back the right chunks for this question?' The transcript emphasizes two implementation rules: reload the persisted vector store correctly, and use the exact same embedding model for the user question that you used for the stored chunks. Mixing embedding models silently breaks retrieval because the vectors no longer share the same space.
The important tunables here are top-k and threshold. Top-k controls how many of the best candidate chunks survive. The similarity threshold controls how strict the retriever is allowed to be. Set the threshold too low and the model gets noisy evidence. Set it too high and the retriever can return nothing at all, even when the answer is present. The point is not to memorize one magic threshold; it is to understand that retrieval quality is a tuning problem with observable tradeoffs.
Operational takeaway: query-time retrieval should always be inspected before blaming generation. If the retrieved chunks already contain the answer, grounding is possible. If they do not, changing the answer prompt alone will not rescue the system.
Exhaustive coverage points to ensure complete topic understanding without missing core concepts.
Query-time retrieval walkthrough: 1) Reload the persisted Chroma store. 2) Embed the user's question with the same embedding model used for the documents. 3) Ask the retriever for the top-k chunks above the score threshold. 4) Inspect the returned chunks before blaming answer quality on the generator. 5) Adjust threshold or top-k only after seeing what evidence the retriever is actually returning. Good generation depends on getting this step right first.
Guided Starter Example
Query-time retrieval walkthrough: 1) Reload the persisted Chroma store. 2) Embed the user's question with the same embedding model used for the documents. 3) Ask the retriever for the top-k chunks above the score threshold. 4) Inspect the returned chunks before blaming answer quality on the generator. 5) Adjust threshold or top-k only after seeing what evidence the retriever is actually returning. Good generation depends on getting this step right first.
Source-grounded Practical Scenario
Query-time retrieval: load the vector store, embed the question, and tune threshold and top-k to return the right chunks.
Source-grounded Practical Scenario
If stored chunks were embedded with one model and the incoming query is embedded with another, the similarity scores become unreliable because the vectors are no longer comparable in the same space.
Second basic RAG example that extends part 1 with additional flow detail.
content/github_code/langchain-course/4_RAGs/1b_basic_part_2.py
Continuation of baseline RAG pipeline.
Open highlighted code →Questions an interviewer is likely to ask about this topic. Think through your answer before reading the senior angle.
Test yourself before moving on. Flip each card to check your understanding — great for quick revision before an interview.
Drag to reorder the architecture flow for RAGs - Basic Example (2). This is designed as an interview rehearsal for explaining end-to-end execution.
Walk through the three practical RAG steps from the LangChain examples: ingest the document once, retrieve chunks with a tuned retriever, and assemble a grounded one-off answer prompt.
Load the persisted vector store, embed the user question with the same embedding model, then tune top-k and threshold until the retriever returns enough relevant context without too much noise.
This mirrors the second basic example: same embedding model, similarity threshold, and top-k control the query-time behavior.
The retriever is returning a compact, relevant candidate set for downstream answer generation.
Question: Where does Gandalf meet Frodo? Documents: [1] It was in the heart of Hobbiton, at the home of Frodo Baggins, that Gandalf came to speak of matters far greater than Frodo could have imagined. [2] Gandalf came to Hobbiton to visit Frodo one summer day, carrying news that would alter the quiet rhythm of the Shire. [3] Frodo was in his home in Hobbiton when Gandalf, his old friend and mentor, arrived for the conversation that changed everything. Instruction: Answer only from the provided documents. If the answer is not found, respond with "I'm not sure".
This request is stateless: retrieve evidence, build one prompt, answer, and stop. That keeps latency and operational complexity lower.
Start flipping cards to track your progress
What is the purpose of basic example 2?
tap to reveal →Improve baseline RAG precision through controlled retrieval and prompt refinements.