Visual walkthrough

DeepSeek V4
KV Cache

A query-centered story: one new token asks, many old tokens answer. DeepSeek V4 changes what gets stored for those old tokens, and how much of that history the query has to read.

QueryKeyValue Shared K/Vc4ac128aLocal
The story

The new query asks the past for help

One decode stepcurrent query in the center
basic attention picture
Q
t0
t1
t2
t3
t4
1. The query

The newest token creates a query vector. It is asking: which old tokens matter right now?

2. The memory

Every old token left behind cached information, so the query does not recompute the whole past.

3. The DeepSeek change

DeepSeek V4 makes each old token leave behind less memory, then summarizes older memory into larger blocks.

Step 1

One token becomes three views

Matrix leveltoken t
x
x Wq
Q
x Wk
K
x Wv
V
Q asks. K matches. V carries content.

score path

Q
dot
K
score(token j) = Q_i dot K_j
then

content path

attention weights
mix
V
output = weighted sum of V_j

In the normal design, K and V are two different learned projections. The cache stores both because future tokens need both paths again.

Step 2

Basic KV cache stores every old K and V

Decode stepnew token t8
basic KV cache view
Q8
K0
V0
K7
V7

In a basic transformer, each old token leaves two memory pieces. The query first uses K pieces to decide relevance, then uses V pieces to collect information.

Q8
looks at
K cache
V cache
returns
O8
Two cached blocks per old token: one K block and one V block.
Step 3

DeepSeek V4 makes one vector do both jobs

Before: separate projections

token 1
token 2
token 3
token 4
x_j -> K_j and V_j

After: shared representation

token 1
token 2
token 3
token 4
x_j -> S_j
What “shared” meansnot pointer sharing; representation sharing
DeepSeek V4 shared-cache view
Q
S0
S1
S2
S3
S4

The query now talks to one shared memory object per old token. That object is still learned by the model; it is not a lossy trick added after training.

Matrix-level answerwhat changed from Wk/Wv?

Traditional KV cache

x_j
Wk
K_j
and Wv
V_j
cache stores: K_j plus V_j

DeepSeek V4 shared cache

x_j
Ws
S_j
cache stores: one shared S_j
Not copied

It is not “make K and reuse it as V” after the fact. The model is trained with a shared cached state.

One cached state

For students, think: two cache-producing matrices become one cache-producing path, `x_j -> S_j`.

Two roles

The same `S_j` is compared with the query to decide relevance, then mixed into the output as content.

as a key

S_j
compared with Q
key-like
score = Q_i dot S_j
same S_j

as a value

S_j
mixed by weights
content-like
output uses the same S_j

The model learns a shared state vector that is useful for matching and content. That is why the cache can store one block per token instead of two.

Step 4

Older shared states are compressed; newest states stay exact

Start hereevery old token first has one shared state S
far past
S0S1S2...S127
middle past
S128S129S130...S255
newest past
S256S257S258S259
Q

What is compressed? The shared states `S`, not the query. DeepSeek V4 has already replaced each old token's separate K and V with one shared state. Compression now groups old `S` states into summary blocks.

C4A: medium compression8 shared states mix into 1 block; next block shifts by 4
shared states before C4A
S0S1S2S3S4S5S6S7
S0S1S2S3S4S5S6S7S8S9S10S11
learned weighted mix
compressed C4A cache
C4A[1]C4A[2]

C4A is not a plain average. The model learns how strongly each `S` state contributes to the summary.

C4A: 8 shared states -> 1 compressed block, then slide forward by 4 states
C128A: heavy compression128 shared states mix into 1 block; next block shifts by 128
shared states before C128A
S0S1S2...S126S127
S128S129S130...S254S255
strong learned mix
compressed C128A cache
C128A[0]C128A[1]

C128A is for far history: it keeps the existence of long-range context without storing every old token exactly.

C128A: 128 shared states -> 1 compressed block, then jump forward by 128 states
The query's memory map after compressionfar = C128A, middle = C4A, newest = exact S
far past: heavily compressed
C128A[0]C128A[1]
middle past: moderately compressed
C4A[63]C4A[64]C4A[65]
newest past: uncompressed
S(t-4)S(t-3)S(t-2)S(t-1)
Q
The newest tokens are deliberately not compressed. They stay exact because local detail matters most near the query.
Step 5

The query now reads a mixed memory map

C4A: selected readingmany C4A blocks -> choose the useful ones
C4A0C4A1C4A2C4A3C4A4C4A5C4A6C4A7
learned indexer scores compressed blocks
C4A0
read
C4A1
skip
C4A2
read
C4A5
read
query reads selected C4A summaries, not every middle-history block
C128A: dense summary readingfew C128A blocks -> read all of them
C128A0C128A1C128A2C128A3Q
already short enough
read
query reads all C128A summaries

C128A compresses so aggressively that the remaining far-history sequence is small. The model can afford to read all of those summaries.

Step 6

The newest tokens stay uncompressed

For the current querycompressed history + short local window
C128A[0] C128A[1] C4A[64] C4A[65] t-5t-4t-3t-2t-1 Q
Q
C128A
C128A
C4A
C4A
local
The query can see summarized long history and exact local detail at the same time.
Step 7

Different layers keep different memory

Layer stackconceptual pattern
L01c4a
L02c128a
L03c4a
L04local
L05c128a
L06c4a
...more
What each layer receivessame tokens, different lens
c4a
c128a
local
Conceptually: c4a layers keep moderately compressed searchable memory, c128a layers keep heavily compressed dense memory, and local layers keep only exact recent tokens.
Final picture

Basic KV cache vs DeepSeek V4 cache

Basic transformer

K
V
Exact K and exact V for every old token.

DeepSeek V4

far
mid
near
Shared K/V, compressed history, exact local window.
Memory ideathe whole story in five moves
1replace K,V with one learned shared state S
2use S for both matching and content
3compress older S tokens with learned weighted pooling
4read C4A sparsely and C128A densely
5keep recent S tokens exact for local detail
Move 1: share K and Vtwo old memory rows become one
K
V
S

Instead of storing a separate key vector and value vector for each old token, the model stores one learned shared vector. That shared vector is trained to serve both roles.

Move 2: compress old shared statesmany S blocks become fewer summary blocks
S tokens
C4A
C128A

C4A is a lighter compression for medium-range history. C128A is a stronger compression for very long-range history.

Move 3: the query sees a mixed memory mapsummaries far away, exact tokens nearby
DeepSeek V4 view
Q
C128
C128
C4
C4
local
Far past

Stored as heavily compressed C128A summaries, so a huge context becomes manageable.

Middle past

Stored as C4A summaries. There are more of them, so the model can select only useful ones.

Recent past

Stored exactly as uncompressed local tokens, because nearby details are often important.

Why this matters

Benefits over a traditional KV cache

Less memory per old token

Traditional cache stores both K and V. DeepSeek V4 stores one shared state first, so every old token starts cheaper.

Older history gets smaller

C4A and C128A turn many shared states into fewer summary blocks, so long contexts do not grow as quickly.

Recent details stay exact

The newest tokens remain uncompressed, so the model still has sharp local memory near the current query.

Memory growth intuitionsame past, different cache shape

Traditional

old t0
old t1
old t2
old t3
linear growth: every token adds K + V

DeepSeek V4

far
mid
near
hybrid growth: summaries far away, exact states nearby
Tradeoff made explicitwhy not compress everything?
far past
C128A

Maximum memory saving.

middle past
C4AC4A

More detail than far summaries.

recent past
SSS

Exact local detail.

Q
The main benefit is not just “smaller cache.” It is a graded memory system: strong compression far away, light compression in the middle, exact memory nearby.