How the machines work

Member lesson

Context windows and what a model forgets

Lesson 10 of 13 in How the machines work.

What is a Context Window? Unlocking LLM Secrets · IBM Technology
Lesson

A language model has no memory. What looks like one is the entire conversation being sent again with every message, which is why long threads get slower and more expensive and why a model that knew your brand voice this morning does not know it tomorrow. The window is large now, and large is still finite.

abottlelitfromthe leftevery word weighs every other wordwhich is why word order and emphasis do anything at all
Five words, each connected to every other, with two connections drawn heavier.
A long paper scroll stretched along a table, spilling over both ends. Two lamps brightly light the start and the end. The middle droops below the table into shadow, where a small bordered card is pinned and hard to see.
The start and the end get the light. A rule buried in the middle gets lost, so move it.
Member

The rest of this is for members.

You have read the opening of this lesson. The rest of it comes with a membership.

One membership covers all four courses and Brands.

  • All lessons in the four Learn courses
  • All brand breakdowns in the Brands section
  • All quizzes, flashcards and crosswords
  • An AI teacher on every lesson, for up to 60 questions a day
  • All course material published during the membership

Price

Membership is from $12 a month on the Yearly plan, which is charged at $144 once a year. Lifetime access is $59, paid once.Membership is from ₹200 a month on the Yearly plan, which is charged at ₹2,400 + GST once a year. Lifetime access is ₹3,999 + GST, paid once.

Check what stuck

Members only

You can't take this check yet.

Members answer these 3 questions and keep the ticket for this lesson. Every lesson has one.

  1. Question 1, locked

    Which of these words means this? “How much text a model can hold at once. Measured in tokens, and it includes its own output.”

    • Head
    • Vision model
    • Latent
    • Context window
  2. Question 2, locked

    Which of these words means this? “Standing instructions set before the conversation. Where a house voice or a brand rule belongs.”

    • Benchmark
    • Embedding
    • Hallucination
    • System prompt
  3. Question 3, locked

    Which of these words means this? “Retrieval-augmented generation. Fetch the relevant passages first, then answer from them.”

    • RAG
    • VAE
    • Compositionality
    • Training

Elsewhere in the library

Hear about new courses.

Leave an email and we will tell you when something new lands. We do not write often.