How to explain attention in transformers
A rigorous interview-ready explanation of transformer self-attention: queries, keys, values, scaling, masking, tradeoffs, and common weak answers.
The short version
Self-attention lets each token build a context-dependent representation by comparing a learned query for that token with learned keys from other tokens, converting the resulting scores into weights, and mixing the corresponding value vectors. Multi-head attention repeats this operation in different learned subspaces. The mechanism provides direct interactions among positions, but ordinary dense attention also carries compute and memory costs that grow quadratically with sequence length.
Why attention was useful
Sequence models need information from other positions to interpret the current position. The Transformer made attention the primary communication mechanism instead of processing tokens only through recurrence or convolution. This allowed much more parallel training while preserving content-dependent interaction among positions.
Queries, keys, and values
Each input representation is projected into a query, key, and value. A query expresses what the current position is looking for; keys expose how other positions can be matched; values contain the information that will be combined if those positions receive weight.
Dot products between queries and keys form relevance scores. Scaling by the square root of key dimension keeps large dot products from pushing softmax into extremely saturated regions. Softmax turns scores into normalized weights, and the weighted sum of values becomes the attention output.
Masking and multiple heads
In causal language modeling, a mask prevents a position from attending to future tokens. Without it, training would leak information unavailable during generation. Multiple heads use separate learned projections, allowing the model to represent different interaction patterns before their outputs are combined.
The boundary of the simple explanation
Attention weights should not automatically be treated as a complete explanation of model reasoning. Dense self-attention compares every position with every other position, producing quadratic score storage and computation in sequence length. During autoregressive inference, a key-value cache avoids recomputing past keys and values, but memory grows with the cached sequence.
Common weak answers
- Calling attention a database lookup without explaining learned similarity and weighted mixing.
- Saying queries, keys, and values are the original tokens rather than learned projections.
- Forgetting scaling, masking, or the sequence-length cost.
- Claiming attention weights directly reveal everything the model considered.
Could you answer this cold?
Explain why a transformer needs separate query, key, and value projections, then describe what causal masking changes and what computational price dense attention pays.
A strong answer should
- Separates matching information from transferred information.
- Explains learned projections and dot-product scoring.
- Connects the causal mask to information available at generation time.
- Names quadratic sequence-length cost without claiming every transformer operation is quadratic.
No account required for the guided preview. Your answer is not placed in the URL.
Primary sources
This guide is an original learning and interview-preparation synthesis. AI assisted with editing; the structure, claims, and cold-answer rubric were reviewed by MyPage.