# GPT-2 **Entity class:** System **Research spine:** [[research/Chronology of Signals, Silos, and Sovereign Software - Research|Chronology of Signals, Silos, and Sovereign Software]] ## Definition GPT-2 is OpenAI's 2019 decoder-only language model, trained to predict the next token across a large WebText corpus. ## Chronological Proximity It followed the 2017 [[wiki/Transformer Architecture|transformer]] and made the historical connection to [[wiki/Claude Shannon Next-Word Generation|Shannon's 1948 procedure]] especially legible. ## Relational Proximity Tokenization, embeddings, masked attention, softmax, sampling, and feedback implement [[wiki/Next-Token Prediction|next-token prediction]] at scale. ## Collection Routes [[collections/The Civilian Intelligence Marketplace|The Civilian Intelligence Marketplace]] · [[collections/Terrorism, Counterterrorism, and the Intelligence Environment|Terrorism, Counterterrorism, and the Intelligence Environment]] · [[wiki/Signals, Silos, and Sovereign Software|Signals, Silos, and Sovereign Software]] ## Sources / Provenance - [[research/Chronology of Signals, Silos, and Sovereign Software - Research|Chronology of Signals, Silos, and Sovereign Software - Research]].