Module 05

Memory, Knowledge, and RAG

Teach the difference between what a model knows, what it sees now, and what it can look up.

← Module 04 Slides for this module Quiz Module 06 →
Lesson 5.1

Short-Term vs Long-Term Memory

Short-term memory helps during the current task, while long-term memory stores useful facts across time.

Simple explanation

In simple terms, Short-term memory helps during the current task, while long-term memory stores useful facts across time. Think of it as giving the system a notebook and a library, not just a memory test. The goal is not to impress you with jargon but to make the idea usable.

Technical translation

Technical translation: Short-Term vs Long-Term Memory usually involves structured inputs, model decisions, and some form of state, tooling, or data flow. Under the hood, engineers turn this idea into repeatable components so the behavior is measurable and easier to improve.

Why it matters

Why it matters: this topic shapes whether an agent feels useful or confusing in real life. When this part is weak, the whole system feels less trustworthy.

Real-world example

A support agent that pulls the latest company policy before answering is using retrieval well.

Mini exercise: Mini exercise: explain short-term vs long-term memory in two sentences as if you were teaching a total beginner, then give one example from daily life or work.
Lesson 5.2

Why Retrieval Matters

Retrieval gives the agent fresh facts so it does not rely only on what was in training.

Simple explanation

A beginner way to understand this is: Retrieval gives the agent fresh facts so it does not rely only on what was in training. If you can explain it to a friend without using buzzwords, you truly understand it.

Technical translation

Technical translation: when builders talk about why retrieval matters, they usually mean a concrete system design with prompts, model calls, validation, and often external tools or data sources. The simple idea stays the same even when the implementation becomes more advanced.

Why it matters

Why it matters: many AI products succeed or fail on this exact idea because it affects quality, cost, user trust, and real-world usefulness.

Real-world example

A support agent that pulls the latest company policy before answering is using retrieval well.

Mini exercise: Mini exercise: pick a tool, app, or task you already know and describe where why retrieval matters shows up inside it.
Lesson 5.3

RAG in Simple Terms

Retrieval-augmented generation means the model first finds useful sources and then answers using them.

Simple explanation

At the simplest level, Retrieval-augmented generation means the model first finds useful sources and then answers using them. It helps to picture the system as a practical helper that follows a clear job instead of as a magic brain.

Technical translation

Technical translation: RAG in Simple Terms usually involves structured inputs, model decisions, and some form of state, tooling, or data flow. Under the hood, engineers turn this idea into repeatable components so the behavior is measurable and easier to improve.

Why it matters

Why it matters: this topic shapes whether an agent feels useful or confusing in real life. When this part is weak, the whole system feels less trustworthy.

Real-world example

A support agent that pulls the latest company policy before answering is using retrieval well.

Mini exercise: Mini exercise: explain rag in simple terms in two sentences as if you were teaching a total beginner, then give one example from daily life or work.
Lesson 5.4

Embeddings Without Math Fear

Embeddings turn meaning into numbers so similar ideas can be found even when wording changes.

Simple explanation

In simple terms, Embeddings turn meaning into numbers so similar ideas can be found even when wording changes. Think of it as giving the system a notebook and a library, not just a memory test. The goal is not to impress you with jargon but to make the idea usable.

Technical translation

Technical translation: when builders talk about embeddings without math fear, they usually mean a concrete system design with prompts, model calls, validation, and often external tools or data sources. The simple idea stays the same even when the implementation becomes more advanced.

Why it matters

Why it matters: many AI products succeed or fail on this exact idea because it affects quality, cost, user trust, and real-world usefulness.

Real-world example

A support agent that pulls the latest company policy before answering is using retrieval well.

Mini exercise: Mini exercise: pick a tool, app, or task you already know and describe where embeddings without math fear shows up inside it.
Lesson 5.5

Vector Databases

A vector database stores embeddings so the agent can quickly fetch related notes, documents, or examples.

Simple explanation

A beginner way to understand this is: A vector database stores embeddings so the agent can quickly fetch related notes, documents, or examples. If you can explain it to a friend without using buzzwords, you truly understand it.

Technical translation

Technical translation: Vector Databases usually involves structured inputs, model decisions, and some form of state, tooling, or data flow. Under the hood, engineers turn this idea into repeatable components so the behavior is measurable and easier to improve.

Why it matters

Why it matters: this topic shapes whether an agent feels useful or confusing in real life. When this part is weak, the whole system feels less trustworthy.

Real-world example

A support agent that pulls the latest company policy before answering is using retrieval well.

Mini exercise: Mini exercise: explain vector databases in two sentences as if you were teaching a total beginner, then give one example from daily life or work.
Lesson 5.6

Knowledge Bases and Notes

A strong knowledge base keeps company facts, workflows, policies, and past decisions easy to reuse.

Simple explanation

At the simplest level, A strong knowledge base keeps company facts, workflows, policies, and past decisions easy to reuse. It helps to picture the system as a practical helper that follows a clear job instead of as a magic brain.

Technical translation

Technical translation: when builders talk about knowledge bases and notes, they usually mean a concrete system design with prompts, model calls, validation, and often external tools or data sources. The simple idea stays the same even when the implementation becomes more advanced.

Why it matters

Why it matters: many AI products succeed or fail on this exact idea because it affects quality, cost, user trust, and real-world usefulness.

Real-world example

A support agent that pulls the latest company policy before answering is using retrieval well.

Mini exercise: Mini exercise: pick a tool, app, or task you already know and describe where knowledge bases and notes shows up inside it.
Lesson 5.7

Keeping Memory Clean

Bad memory makes agents worse, so stored facts should be accurate, durable, and worth keeping.

Simple explanation

In simple terms, Bad memory makes agents worse, so stored facts should be accurate, durable, and worth keeping. Think of it as giving the system a notebook and a library, not just a memory test. The goal is not to impress you with jargon but to make the idea usable.

Technical translation

Technical translation: Keeping Memory Clean usually involves structured inputs, model decisions, and some form of state, tooling, or data flow. Under the hood, engineers turn this idea into repeatable components so the behavior is measurable and easier to improve.

Why it matters

Why it matters: this topic shapes whether an agent feels useful or confusing in real life. When this part is weak, the whole system feels less trustworthy.

Real-world example

A support agent that pulls the latest company policy before answering is using retrieval well.

Mini exercise: Mini exercise: explain keeping memory clean in two sentences as if you were teaching a total beginner, then give one example from daily life or work.
Lesson 5.8

When Not to Use RAG

If the answer is already in the prompt or the task is simple, extra retrieval can add cost and confusion.

Simple explanation

A beginner way to understand this is: If the answer is already in the prompt or the task is simple, extra retrieval can add cost and confusion. If you can explain it to a friend without using buzzwords, you truly understand it.

Technical translation

Technical translation: when builders talk about when not to use rag, they usually mean a concrete system design with prompts, model calls, validation, and often external tools or data sources. The simple idea stays the same even when the implementation becomes more advanced.

Why it matters

Why it matters: many AI products succeed or fail on this exact idea because it affects quality, cost, user trust, and real-world usefulness.

Real-world example

A support agent that pulls the latest company policy before answering is using retrieval well.

Mini exercise: Mini exercise: pick a tool, app, or task you already know and describe where when not to use rag shows up inside it.