# Model Error **Domain:** Modeling / Statistics / Governance **Doc Type:** Failure-Mode Node **Maturity:** Developed ## Definition **Model error** is consequential mismatch between a model and the system, process or subject it is used to represent. ## Simple Reminders, Quotations, and Thoughts > "What I think about machines thinking is that it won't happen anytime soon. I don't imagine that there is any in-principle limitation; carbon isn't magical, and I suspect silicon will do just fine." > **— Gary Marcus**, *2015, Edge annual question “What Do You Think About Machines That Think?”* [[reminders/Machine Succession/Carbon Is Not Magical but Thinking Machines Are Not Near by Gary Marcus|Carbon Is Not Magical but Thinking Machines Are Not Near by Gary Marcus]] > "Taken together there is nothing like "intelligence" which can be extracted as a precise concept and which can be used as a reference for "artificial intelligence"." > **— Ernst Pöppel**, *2015, Edge annual question “What Do You Think About Machines That Think?”* [[reminders/Risk Debate/Intelligence May Be Too Imprecise to Artificially Reproduce by Ernst Pöppel|Intelligence May Be Too Imprecise to Artificially Reproduce by Ernst Pöppel]] > "Machine intelligence, while impressive in certain areas, is still narrow and inflexible." > **— Timo Hannay**, *2015, Edge annual question “What Do You Think About Machines That Think?”* [[reminders/Machine Succession/Machine Intelligence Is Still Narrow and Inflexible by Timo Hannay|Machine Intelligence Is Still Narrow and Inflexible by Timo Hannay]] > "A human player can make generalizations and describe why certain types of moves are good, and use that to teach a human player." > **— Rodney A. Brooks**, *2015, Edge annual question “What Do You Think About Machines That Think?”* [[reminders/Risk Debate/Mistaking Performance For Competence Misleads Estimates Of AIs 21st Century Promise And Danger by Rodney A. Brooks|Mistaking Performance For Competence Misleads Estimates Of AIs 21st Century Promise And Danger by Rodney A. Brooks]] > "Most such prophecies are grounded in a false analogy between human nature and computer nature, or natural intelligence and artificial intelligence." > **— Michael Shermer**, *2015, Edge annual question “What Do You Think About Machines That Think?”* [[reminders/Machine Succession/When It Comes To AI, Think Protopia, Not Utopia Or Dystopia by Michael Shermer|When It Comes To AI, Think Protopia, Not Utopia Or Dystopia by Michael Shermer]] > "Building models is very different from proclaiming truths. It's a never-ending process of discovery and refinement, not a war to win or destination to reach. Uncertainty is intrinsic to the process of finding out what you don't know, not a weakness to avoid. Bugs are features — violations of expectations are opportunities to refine them. And decisions are made by evaluating what works better, not by invoking received wisdom." > **— Neil Gershenfeld**, *2011, Edge Annual Question, “What Scientific Concept Would Improve Everybody's Cognitive Toolkit?”* [[reminders/Information/Uncertainty Is a Feature of Discovery by Neil Gershenfeld|Uncertainty Is a Feature of Discovery by Neil Gershenfeld]] ## Sources of Error Error can arise from wrong structure, omitted variables, poor calibration, stale data, measurement bias, invalid transfer between contexts or use outside the model's intended purpose. ## Governing Variable Model error governs the difference between apparent and actual decision conditions. ## Relation to Governance An error becomes politically important when an [[wiki/Operational Representation|operational representation]] allocates access or changes the environment. [[wiki/Continuous Reconciliation|Continuous Reconciliation]] can amplify the problem if correction based on the wrong model generates new observations that appear to confirm it. ## Constitutional Question and Failure Modes Systems should expose uncertainty, intended use, provenance and evidence boundaries. The worst failure is not simply an inaccurate prediction but a model that becomes unappealable because its output is treated as the authoritative reality. ## Related Ontology [[wiki/Model Fidelity|Model Fidelity]] · [[wiki/Evidence Boundary|Evidence Boundary]] · [[wiki/Model-Subject Boundary|Model-Subject Boundary]] · [[wiki/Representational Due Process|Representational Due Process]] ## Relationships - **Edge Annual Question source relationship:** [[collections/Edge|Edge]] preserves the annual-question source corpus from which a proposition-specific quotation is connected to this entry.