AI production systems

Member lesson

Caching, and not paying twice

Lesson 4 of 10 in AI production systems.

What is Prompt Caching? Optimize LLM Latency with AI Transformers · IBM Technology
Lesson

Generation with fixed inputs asks for the same result every time, which makes it unusually cacheable. The same model, prompt, seed and settings request the same image, so a system that asks twice is paying twice for something it already has.

A pantry shelf of sealed jars, each holding a rolled print with a tag showing small drawings of a lens, a seed pod, a lamp and a gear. A hand reaches for one jar while a stove across the room stays unlit.
Same recipe, same picture. Take it from the jar, and make sure the tag lists every ingredient.
Member

The rest of this is for members.

You have read the opening of this lesson. The rest of it comes with a membership.

One membership covers all four courses and Brands.

  • All lessons in the four Learn courses
  • All brand breakdowns in the Brands section
  • All quizzes, flashcards and crosswords
  • An AI teacher on every lesson, for up to 60 questions a day
  • All course material published during the membership

Price

Membership is from $12 a month on the Yearly plan, which is charged at $144 once a year. Lifetime access is $59, paid once.Membership is from ₹200 a month on the Yearly plan, which is charged at ₹2,400 + GST once a year. Lifetime access is ₹3,999 + GST, paid once.

Check what stuck

Members only

You can't take this check yet.

Members answer these 3 questions and keep the ticket for this lesson. Every lesson has one.

  1. Question 1, locked

    Which of these words means this? “A stored answer, reused instead of recomputed.”

    • Exit
    • Trigger
    • Cache
    • API
  2. Question 2, locked

    Which of these words means this? “What decides whether two requests are the same. Getting it wrong is the whole risk.”

    • Invalidation
    • Cache key
    • Assertion
    • Vision check
  3. Question 3, locked

    Which of these words means this? “Knowing when a cached answer has gone stale. Famously the hard part.”

    • Invalidation
    • Threshold
    • CDN
    • Retention

Elsewhere in the library

Hear about new courses.

Leave an email and we will tell you when something new lands. We do not write often.