# Transformer Architecture **Entity class:** Technology **Research spine:** [[research/Chronology of Signals, Silos, and Sovereign Software - Research|Chronology of Signals, Silos, and Sovereign Software]] ## Definition The transformer is a neural-network architecture that uses attention to relate positions in a sequence and, in decoder-only form, masks future tokens during generation. ## Chronological Proximity It was introduced in 2017 and rapidly became the basis of [[wiki/GPT-2|GPT-2]] in 2019 and later large language models. ## Relational Proximity [[wiki/Attention Is All You Need|Attention Is All You Need]] established the architecture; [[wiki/Next-Token Prediction|next-token prediction]] supplies the training objective and [[wiki/Temperature|temperature]] shapes sampling. ## Collection Routes [[collections/The Civilian Intelligence Marketplace|The Civilian Intelligence Marketplace]] · [[collections/Terrorism, Counterterrorism, and the Intelligence Environment|Terrorism, Counterterrorism, and the Intelligence Environment]] · [[wiki/Signals, Silos, and Sovereign Software|Signals, Silos, and Sovereign Software]] ## Sources / Provenance - [[research/Chronology of Signals, Silos, and Sovereign Software - Research|Chronology of Signals, Silos, and Sovereign Software - Research]].