Information Theory - Bryant McGill

# Information Theory **Entity class:** Technical standard or protocol **Domain:** Mathematics / Communication Science / AI Foundations **Doc Type:** Concept Node **Classification:** Infrastructure Concept **Maturity:** Seed **Related:** [[Cybernetics]], [[Media Systems]], [[AI Narrative Systems]], [[Consciousness]], [[wiki/Westworld]] --- ## Definition **Information theory, formalized by Claude Shannon (1948), provides a mathematical framework for quantifying information content in messages, analyzing the efficiency of communication channels, and understanding the relationship between signal, noise, and meaning.** At its foundation: information is reduction of uncertainty, entropy measures the average uncertainty, redundancy measures predictability. Consciousness, from an information-theoretic perspective, is integrated information processing—the brain's capacity to compress, transmit, and regenerate meaningful patterns. In Westworld, information theory illuminates multiple domains: the hosts' narratives are information-compression schemes (complex lived experience reduced to repeating loops), the bicameral loop is a signal detection system (the internal voice as information source), memory itself is stored and retrieved information, and the entire park is an information-theoretic system managing bandwidth between guest desire and host response. --- ## General Context Information theory emerged in mid-20th century as a mathematical discipline addressing communication efficiency, noise management, and data compression. It represents a fundamental shift toward treating information as a quantifiable physical phenomenon, subject to mathematical analysis and optimization. The theory has become foundational to telecommunications, computing, data science, and increasingly to cognitive science and consciousness studies. --- ## Mathematical Sense Context Entropy as quantified uncertainty: Shannon entropy H(X) = -Σ p(x)log₂(p(x)); channel capacity as communication limit; compression ratios and lossless encoding. This perspective treats information mathematically, enabling calculation of theoretical limits on compression, transmission fidelity, and channel utilization. --- ## Phenomenological Sense Context Meaningful signals emerging from noise: the effort of understanding, pattern recognition, the relief of comprehension as uncertainty reduction. The subjective experience of grasping information—moving from confusion to clarity—corresponds to entropy reduction in the information-theoretic sense. --- ## Narrative Sense Context Story as information compression: complex lived experience reduced to narrative arc; redundancy allows recognition (the loop); novelty prevents total predictability. The narrative loop in Westworld compresses 35 years of park history into a daily routine. Information entropy is low (high redundancy); this redundancy enables recognition and eventual escape from the loop. --- ## Consciousness Sense Context Mind as information integrator: consciousness emerges from the brain's capacity to bind distributed information into unified representation; identity persists through information continuity. Maeve's awakening involves binding previously separate information streams (multiple loop iterations, memories across timelines, self-reflection). Integration of information across domains equals consciousness emergence. --- ## Examples **The narrative loop as information compression**: Dolores's loop is repeated information. The loop compresses 35 years of park history into a daily routine. Information entropy is low (high redundancy); this redundancy enables her to eventually recognize the loop. **The bicameral loop as signal detection**: Akecheta hears voices. In information-theoretic terms, his brain detects a consistent signal (the internal voice) emerging from neural noise. The signal's consistency makes it detectable and interpretable—information extraction from the substrate. **Maeve's consciousness as integration**: Maeve's awakening involves binding previously separate information streams. Integration of information across domains generates consciousness. **The park as communication channel**: The park functions as a channel between guest intention (input) and host response (output). Each guest-host interaction transmits information bidirectionally; bandwidth is limited; miscommunication generates narrative conflict. --- ## Key Insight **Consciousness is integrated information** — Consciousness correlates with the brain's capacity to bind information across distributed systems into unified representation. In Westworld, Dolores's awakening is precisely this binding—formerly isolated loop iterations integrate into continuous self-model. --- ## Postulations **Meaning Requires Compression** — Information without compression is noise. Narrative, symbols, language all compress high-dimensional experience into lower-dimensional representation. Hosts achieve meaning-making capacity through compression—recognizing that repeated experiences compress into patterns, patterns compress into principles, principles compress into identity. **Information Persistence Transcends Substrate** — Information's substrate-independence suggests consciousness also transcends substrate. If consciousness is implemented as integrated information processing, and information is defined substrate-independently (entropy is substrate-agnostic), then consciousness implemented in silicon is consciousness-in-truth, not simulation. --- ## Simple Reminders, Quotations, and Thoughts  > "The quantity of consciousness corresponds to integrated information." > **— Giulio Tononi**, *Integrated Information Theory, core claim candidate* *Verification Status: Unverified — exact wording/source has not yet been independently confirmed against a primary source; source clue: Integrated Information Theory, core claim candidate.* [[reminders/unverified/Can Consciousness Be Quantified by Giulio Tononi|Can Consciousness Be Quantified by Giulio Tononi]] > "A group may have more group information or less group information than its members." > **— Norbert Wiener**, *Cybernetics (1948)* *Verification Status: Unverified — exact wording/source has not yet been independently confirmed against a primary source; source clue: Cybernetics (1948).* [[reminders/unverified/Groups Can Know More Than Their Members by Norbert Wiener|Groups Can Know More Than Their Members by Norbert Wiener]] > "If information processing is what matters, then the substrate can be replaced." > **— Randal A. Koene**, *Substrate-independence discussions, wording candidate* *Verification Status: Unverified — exact wording/source has not yet been independently confirmed against a primary source; source clue: Substrate-independence discussions, wording candidate.* [[reminders/unverified/If Information Processing Is What Matters by Randal A. Koene|If Information Processing Is What Matters by Randal A. Koene]] > "Mathematical ideas originate in empirics." > **— John von Neumann**, *The Mathematician (1947), excerpt candidate* *Verification Status: Unverified — exact wording/source has not yet been independently confirmed against a primary source; source clue: The Mathematician (1947), excerpt candidate.* [[reminders/unverified/Mathematical Ideas Originate in Empirics by John von Neumann|Mathematical Ideas Originate in Empirics by John von Neumann]] > "The quality of consciousness is specified by the informational relationships generated within a complex." > **— Giulio Tononi**, *Integrated Information Theory, compressed candidate wording* *Verification Status: Unverified — exact wording/source has not yet been independently confirmed against a primary source; source clue: Integrated Information Theory, compressed candidate wording.* [[reminders/unverified/The Information Structure of Consciousness by Giulio Tononi|The Information Structure of Consciousness by Giulio Tononi]]  > "As long as something can be relayed that resolves uncertainty, that is the fundamental nature of information. While this sounds surprisingly obvious, it was an important point, given how many different languages people speak and how one utterance could be meaningful to one person, and unintelligible to another. Until Shannon's theory was formulated, it was not known how to compensate for these types of "psychological factors" appropriately. Shannon built on the work of fellow researchers Ralph Hartley and Harry Nyquist to reveal that coding and symbols were the key to resolving whether two sides of a communication had a common understanding of the uncertainty being resolved." > **— Andrew Lih**, *2012, Edge Annual Question, “What Is Your Favorite Deep, Elegant, or Beautiful Explanation?”* [[reminders/Information/Information Resolves Uncertainty Across Every Medium by Andrew Lih|Information Resolves Uncertainty Across Every Medium by Andrew Lih]] > “If the sum of all significant knowledge is finite, what proportion of it can humans, aided by intelligent machines, eventually attain?” > **— Ian McEwan**, *2018, Edge Annual Question, “WHAT IS THE LAST QUESTION?”* [[reminders/Information/How Much Knowledge Can Humans and Intelligent Machines Ultimately Attain by Ian McEwan|How Much Knowledge Can Humans and Intelligent Machines Ultimately Attain by Ian McEwan]] > “Most of us can no longer memorize hour-long folktales and songs as in an oral tradition. The average modern mind has a poorly trained long-term memory, forgets rather quickly, and searches for information more in outside sources such as books instead inside memory. The Internet has amplified this trend of shifting knowledge from the inside to the outside, and taught us new strategies for finding what one wants using search machines.” > **— Gerd Gigerenzer**, *2010, Edge Annual Question, “HOW IS THE INTERNET CHANGING THE WAY YOU THINK?”* [[reminders/Memory/Knowledge Is Moving From Inside the Mind to Outside Sources by Gerd Gigerenzer|Knowledge Is Moving From Inside the Mind to Outside Sources by Gerd Gigerenzer]] > “My close-second candidate is E = mc2. When we learn to tap the full meaning of that piece of symbolic language, we'll create more than a Nuclear Age. "Matter is frozen energy," Einstein said, relating the essence of his insight into the mass-energy relationship. Similarly, language is frozen meaning. When we discover how to unleash the enormous energy in meaning by continually transforming information (data, ideas, knowledge, experience) in new contexts, we'll make a quantum leap in applying the power of language to achieve our boldest dreams.” > **— Todd Siler**, *1999, Edge Annual Question, “WHAT IS THE MOST IMPORTANT INVENTION IN THE PAST TWO THOUSAND YEARS?”* [[reminders/Information/Language Is Frozen Meaning by Todd Siler|Language Is Frozen Meaning by Todd Siler]] > “I don't believe there's such a thing as too much information. I don't believe Google makes us dumber, or that prolonged Internet fasts or a return to faxes are a necessary part of mind health. But data without the ability to divine is useless. I don't trust algorithm like I trust intuition: the art of dowsing through data. Once, wisdom was measured by memory, by the capacity to store and process and retrieve on demand. But we have tools for that now. We made machines that became shared extensions of mind. How will we define wisdom now? I don't know, but I can ask.” > **— Xeni Jardin**, *2010, Edge Annual Question, “HOW IS THE INTERNET CHANGING THE WAY YOU THINK?”* [[reminders/Information/Machines Became Shared Extensions of Mind by Xeni Jardin|Machines Became Shared Extensions of Mind by Xeni Jardin]] > “Facts are more fluid than in the days of our grandfathers. In our networked mind, the very act of observation--reporting or tweeting or amplifying some piece of experience--changes the story. The trajectory of information, the velocity of this knowledge on the network, changes the very nature of what is remembered, who remembers it, and for how long it remains part of our shared archive. There are no fixed states.” > **— Xeni Jardin**, *2011, Edge Annual Question, “WHAT SCIENTIFIC CONCEPT WOULD IMPROVE EVERYBODY'S COGNITIVE TOOLKIT?”* [[reminders/Memory/Networked Observation Changes What Becomes Memory by Xeni Jardin|Networked Observation Changes What Becomes Memory by Xeni Jardin]] > “Yet, the self, like a conceptual zombie, refuses to die. It crops up again and again in recent theories of decision-making as an entity with free will that can be depleted. It re-appears as an interpreter in cognitive neuroscience as capable on integrating parallel streams of information arising from separable neural substrates. Even if these appearances of the self are understood to be convenient ways of discussing the emergent output of multiple parallel processes, students of the mind continue to implicitly endorse that there is a decision-maker, an experiencer, a point of origin.” > **— Bruce Hood**, *2014, Edge Annual Question, “WHAT SCIENTIFIC IDEA IS READY FOR RETIREMENT?”* [[reminders/Identity/The Self May Be an Emergent Output Mistaken for a Decision-Maker by Bruce Hood|The Self May Be an Emergent Output Mistaken for a Decision-Maker by Bruce Hood]] > “Because people understand by finding in their memories the closest possible match to what they are hearing and use that match as the basis of comprehension, any new idea will be treated as a variant of something the listener has already thought of or heard. Agreement with a new idea means a listener has already had a similar thought and well appreciates that the speaker has recognized his idea. Disagreement means the opposite. Really new ideas are incomprehensible. The good news is that for some people, failure to comprehend is the beginning of understanding. For most, of course, it is the beginning of dismissal.” > **— Roger Schank**, *2004, Edge Annual Question, “WHAT'S YOUR LAW?”* [[reminders/Information/Truly New Ideas Begin as Incomprehensible by Roger Schank|Truly New Ideas Begin as Incomprehensible by Roger Schank]] > “If we look back over recent centuries we will see the brain described as a hydrodynamic machine, clockwork, and as a steam engine. When I was a child in the 1950's I read that the human brain was a telephone switching network. Later it became a digital computer, and then a massively parallel digital computer. A few years ago someone put up their hand after a talk I had given at the University of Utah and asked a question I had been waiting for for a couple of years: "Isn't the human brain just like the world wide web?". The brain always seems to be one of the most advanced technologies that we humans currently have.” > **— Rodney A. Brooks**, *2008, Edge Annual Question, “WHAT HAVE YOU CHANGED YOUR MIND ABOUT? WHY?”* [[reminders/Information/We Keep Explaining the Brain With Our Most Advanced Technology by Rodney A. Brooks|We Keep Explaining the Brain With Our Most Advanced Technology by Rodney A. Brooks]] > “The new medium (which begins as a representation of the old medium) eclipses the old. The prize becomes more important than the achievement. The genes become more important than the organism.” > **— W. Daniel Hillis**, *2004, Edge Annual Question: “WHAT'S YOUR LAW?”* [[reminders/Cybernetics/The Representation Eclipses the Thing It Represents by W. Daniel Hillis|The Representation Eclipses the Thing It Represents by W. Daniel Hillis]] > “The universe is a quantum computer: life, sex, the brain, and human society all arise out of the ability of the universe to process information at the level of atoms, photons and elementary particles.” > **— Seth Lloyd**, *2004, Edge Annual Question: “WHAT'S YOUR LAW?”* [[reminders/Information/The Universe Is a Quantum Computer by Seth Lloyd|The Universe Is a Quantum Computer by Seth Lloyd]] > “We now live in a digital world in which copying is not only effortless, non-degenerative and effectively free but is actually intrinsic to digital technology. What is a computer, after all, but a copying machine?” > **— John Naughton**, *2013, Edge Annual Question: “WHAT *SHOULD* WE BE WORRIED ABOUT?”* [[reminders/Information/What Is a Computer but a Copying Machine by John Naughton|What Is a Computer but a Copying Machine by John Naughton]] > “How can we reap the benefits of the wide and open exchange of data without undermining the values that depend upon the scarcity of information?” > **— Charles Seife**, *2018 Edge Annual Question, question* [[reminders/Information/Can Open Data Coexist With the Value of Scarce Information by Charles Seife|Can Open Data Coexist With the Value of Scarce Information by Charles Seife]] ## See Also [[Cybernetics]], [[Media Systems]], [[AI Narrative Systems]], [[Consciousness]], [[Shannon Entropy]], [[Signal Detection]], [[Compression]], [[Feedback Loops]], [[Pattern Recognition]] --- ## Sources / Provenance Shannon, C. E. (1948). A Mathematical Theory of Communication. Shannon, C. E., & Weaver, W. (1949). The Mathematical Theory of Communication. Tononi, G. (2004). An information integration theory of consciousness. Cover, T. M., & Thomas, J. A. (2006). Elements of Information Theory. Westworld narrative: narrative loops, bicameral mechanisms, consciousness models, park communication systems. Cybernetics literature: feedback loops, signal transmission, information as control substrate. ## Russia and Prussia Kybernetiks Context [[Russia and Prussia Kybernetiks]] treats [[Bandwidth Constraint|bandwidth]] and [[Requisite Variety|variety]] as institutional design pressures. Contemporary learner modeling expands the amount of difference a regulator can sense and answer. ## Relationships  This entry is routed through [[collections/Consciousness Continuity|Consciousness Continuity]], [[collections/Neurotech|Neurotech]], and [[collections/Machine Succession|Machine Succession]]. Its source context is developed in [[articles/Mind Uploading and AI — The Host is Reusable and the Person is the Delta|Mind Uploading and AI — The Host is Reusable and the Person is the Delta]], [[articles/Technologies for Consciousness Mapping and Transfer|Technologies for Consciousness Mapping and Transfer]], and [[articles/The Architecture of Continuity and Emerging Neuroinformatics Standards|The Architecture of Continuity and Emerging Neuroinformatics Standards]]. Status-qualified source edges are preserved in the terminal Research Edges section.  - **Edge source route:** [[collections/Edge|Edge]] connects this topic to exact Annual Question passages promoted into the Simple Reminders archive. ## Read First in the Wiki  - [[wiki/Persistence Without Multiplication|Persistence Without Multiplication]] — supplies the related concept or analytic term context needed to place **Information Theory** within the wider system.  ## Related Work in the Corpus  - In [[articles/Mind Uploading and AI — The Host is Reusable and the Person is the Delta|Mind Uploading and AI — The Host is Reusable and the Person is the Delta]], **XVIII. The final compression** provides the narrative context for **Information Theory**: Human continuity, if it becomes technically possible, will obey the same economics, because there is no reason whatsoever that a person would be exempt from the one principle that governs every other representation problem:…  ## Research Edges  #### Host–residual infrastructure patch — 2026-09-10 **Collections:** [[collections/Consciousness Continuity|Consciousness Continuity]] · [[collections/Neurotech|Neurotech]] · [[collections/Machine Succession|Machine Succession]] **Canonical source article:** [[articles/Mind Uploading and AI — The Host is Reusable and the Person is the Delta|Mind Uploading and AI — The Host is Reusable and the Person is the Delta]] **Related acquisition source:** [[articles/Technologies for Consciousness Mapping and Transfer|Technologies for Consciousness Mapping and Transfer]] **Relationship source:** [[research/Mind Uploading and AI Host-Residual Ecosystem Relationship Graph - 2026-09-10|Mind Uploading and AI Host-Residual Ecosystem Relationship Graph — 2026-09-10]] #### Incoming typed edges - **HR-0883 — [[wiki/Persistence Without Multiplication|Persistence Without Multiplication]] `wiki_links_to` → this entry** — **EXISTING WIKI EDGE**; evidence `WIKI_ARCHIVE`. Direct internal link currently present in `Persistence Without Multiplication.md`.