# Information Theory **Domain:** Mathematics / Communication Engineering / Computation **Doc Type:** Concept Node **Maturity:** Developed ## Definition **Information theory** is the mathematical framework for quantifying uncertainty, representing messages, coding data, and determining the limits of reliable communication. [[wiki/Claude Shannon|Claude Shannon]] established the field in his 1948 paper [[wiki/A Mathematical Theory of Communication|A Mathematical Theory of Communication]], written at [[wiki/Bell Labs|Bell Labs]]. Its core concepts—bit, entropy, source coding, channel capacity, noise, and error-correcting redundancy—underlie digital communication, compression, storage, cryptography, and parts of statistical learning. The framework applies wherever a process can be modeled as selection and transmission among possible messages; that generality does not make semantic meaning, biological function, or consciousness identical to information quantity. ## Modern routes - [[wiki/Cybernetics|Cybernetics]] joins communicated information to feedback and control. - [[wiki/Biological Cybernetics|Biological Cybernetics]] applies information-and-control models to nervous systems, perception, action, and adaptation. - [[wiki/Brain-Computer Interfaces|Brain-Computer Interfaces]] expose practical questions of channel capacity, signal-to-noise ratio, coding, decoder uncertainty, and closed-loop feedback. - [[wiki/Machine Intelligence Continuum|Machine Intelligence Continuum]] traces the role of information theory in engineered cognition. - [[wiki/Information, Physics, Quantum|Information, Physics, Quantum]] and [[wiki/Quantum Foundations|Quantum Foundations]] extend or interrogate the role of information in physical theory. ## Relationships - **Foundational paper:** [[wiki/A Mathematical Theory of Communication|A Mathematical Theory of Communication]]. - **Founder and institution:** [[wiki/Claude Shannon|Claude Shannon]] at [[wiki/Bell Labs|Bell Labs]]. - **Engineering-to-biology bridge:** [[wiki/Biological Cybernetics|Biological Cybernetics]] and [[wiki/Max Planck Institute for Biological Cybernetics|Max Planck Institute for Biological Cybernetics]]. - **Neurotechnology route:** [[collections/Neurotech|Neurotech]], [[wiki/Neural Interface|Neural Interface]], and [[wiki/Brain-Computer Interfaces|Brain-Computer Interfaces]]. - **Article context:** [[articles/Bell Labs and the Distributed Architecture of American Power|Bell Labs and the Distributed Architecture of American Power]]. ## Simple Reminders, Quotations, and Thoughts > "Knowledge itself is power." > **— Francis Bacon**, *1597, Meditationes Sacrae* *Verification Status: Unverified — exact wording/source has not yet been independently confirmed against a primary source; source clue: Meditationes Sacrae, 1597.* [[reminders/unverified/Knowledge Itself Is Power by Francis Bacon|Knowledge Itself Is Power by Francis Bacon]] > "Life is a DNA software system." > **— J. Craig Venter**, *2010, Synthetic genomics talk / interview* *Verification Status: Unverified — exact wording/source has not yet been independently confirmed against a primary source; source clue: Synthetic genomics talk / interview, 2010.* [[reminders/unverified/Life Is a DNA Software System by J. Craig Venter|Life Is a DNA Software System by J. Craig Venter]] > "Science is what we understand well enough to explain to a computer. Art is everything else we do." > **— Donald E. Knuth**, *1996, Foreword to A=B* *Verification Status: Unverified — exact wording/source has not yet been independently confirmed against a primary source; source clue: Foreword to A=B, 1996.* [[reminders/unverified/Science Is What We Understand Well Enough to Explain to a Computer Art Is Everything Else We Do by Donald E. Knuth|Science Is What We Understand Well Enough to Explain to a Computer Art Is Everything Else We Do by Donald E. Knuth]] > "The Internet is a reflection of our society and that mirror is going to be reflecting what we see." > **— Vint Cerf**, *Public talks on Internet governance* *Verification Status: Unverified — exact wording/source has not yet been independently confirmed against a primary source; source clue: Public talks on Internet governance.* [[reminders/unverified/The Internet Is a Reflection of Our Society and That Mirror Is Going to Be Reflecting What We See by Vint Cerf|The Internet Is a Reflection of Our Society and That Mirror Is Going to Be Reflecting What We See by Vint Cerf]] > "We are all walking around with essentially the same instruction book, with just a few spelling differences." > **— Francis Collins**, *Human Genome Project public explanation* *Verification Status: Unverified — exact wording/source has not yet been independently confirmed against a primary source; source clue: Human Genome Project public explanation.* [[reminders/unverified/We Are All Walking Around with Essentially the Same Instruction Book with Just a Few Spelling Differences by Francis Collins|We Are All Walking Around with Essentially the Same Instruction Book with Just a Few Spelling Differences by Francis Collins]] ## Sources / Provenance - Claude E. Shannon, [“A Mathematical Theory of Communication,” part I](https://doi.org/10.1002/j.1538-7305.1948.tb01338.x) and [part II](https://doi.org/10.1002/j.1538-7305.1948.tb00917.x), *Bell System Technical Journal* 27 (1948). - [[articles/Bell Labs and the Distributed Architecture of American Power|Bell Labs and the Distributed Architecture of American Power]].