Find a concept

Search Notes and Discovery

Enter a term to search published Notes and Discovery.

    In this topic

    Structured note

    Positional encoding and the encoder-decoder

    Where position, masks, residual paths, encoder blocks, and decoder blocks fit in a complete Transformer.

    TransformersIntermediate
    Published
    Reviewed
    Reading time
    3 min

    Prerequisites

    On this page

    Direct answer

    A Transformer combines token representations with positional information, then repeatedly applies attention and a position-wise feed-forward network through residual paths and normalization. In an encoder-decoder model, the encoder builds contextual source representations; the decoder reads earlier target tokens and attends to the encoder while producing the next-token distribution.

    Why position must be added

    Self-attention compares content at all positions but, by itself, does not encode whether a token came first, last, or three steps away. Position information breaks that symmetry.

    The original Transformer adds fixed sine and cosine signals to token embeddings:

    PE(pos,2i)=sin(pos/100002i/dmodel)PE_{(pos,2i)}=\sin\left(pos/10000^{2i/d_{model}}\right) PE(pos,2i+1)=cos(pos/100002i/dmodel)PE_{(pos,2i+1)}=\cos\left(pos/10000^{2i/d_{model}}\right)

    Other valid designs include learned absolute embeddings, relative-position biases, and rotary position embeddings. They are not interchangeable implementation details: each changes how position enters attention and may behave differently beyond trained sequence lengths.

    Encoder block

    The original encoder repeats two sublayers:

    1. Multi-head self-attention, where every non-padding source position can read other source positions.
    2. A feed-forward network applied independently at each position.

    Each sublayer sits on a residual path and is paired with normalization. The feed-forward network changes features within each token; attention moves information between tokens.

    After the final encoder block, the output acts as memory for decoder cross-attention. It contains contextual source representations rather than a single compressed sentence vector.

    Decoder block

    The original decoder repeats three sublayers:

    1. Masked self-attention over the target prefix.
    2. Cross-attention whose queries come from the decoder and whose keys and values come from encoder output.
    3. A position-wise feed-forward network.

    The causal mask is essential during autoregressive training. Position (t) may use target tokens up to (t), but not later ground-truth tokens. Without that mask, training would leak the answer the model is supposed to predict.

    End-to-end information flow

    StageReads fromProduces
    Source embedding + positionSource token IDsPosition-aware source vectors
    Encoder stackWhole unmasked sourceContextual encoder memory
    Target embedding + positionKnown target prefixPosition-aware target vectors
    Masked decoder attentionEarlier target positionsPrefix context
    Cross-attentionEncoder memorySource-conditioned decoder state
    Output projectionFinal decoder stateNext-token logits

    Encoder-only, decoder-only, and encoder-decoder models keep different subsets of this blueprint. A decoder-only language model has causal self-attention but no separate encoder or cross-attention. Calling every Transformer “encoder-decoder” hides that distinction.

    Limitations and design choices

    Dense attention still has a quadratic score matrix in sequence length. Fixed absolute positions can extrapolate poorly in some settings; newer methods address position differently but introduce their own assumptions. Residual and normalization placement also varies across model families, so diagrams should name the architecture they describe rather than presenting one order as universal.

    Interview takeaway

    Position makes order available; encoder self-attention builds source memory; masked decoder self-attention preserves causality; and cross-attention connects generated tokens to the encoded source.

    Sources

    1. Attention Is All You Need
    2. RoFormer: Enhanced Transformer with Rotary Position Embedding