Language · 2017 · sequence transduction
Encoder–Decoder
Two coupled transformers: an encoder reads the full input bidirectionally; a decoder generates the output conditioned on that reading.
ModLensLanguage
Interactive Diagram
Focus the lens
Click any component to read what it does. Signal direction follows the edges.
inputSource. The complete input, read in both directions.
Core idea
Separate understanding from production. The encoder builds a complete representation of the source; the decoder attends to it while emitting the target.
Why it exists
Translation, summarization and structured generation need full-source understanding plus incremental production — two distinct computations.
Data Flow
What moves through the system
- 01Encoder reads the entire input with bidirectional attention.
- 02Decoder generates tokens autoregressively.
- 03Cross-attention lets the decoder query the encoder's representation.
- 04Output sequence grows until a stop condition.
Strengths
- + Natural fit for transduction tasks
- + Bidirectional source understanding
Limitations
- − Two stacks to train and serve
- − General-purpose assistants settled on decoder-only stacks; encoder–decoder remains the specialist's tool for transduction
Applications
- · Machine translation
- · Summarization
- · Speech recognition
- · Structured extraction