# **Harms Incurred While Bringing Preventive Systems Online** <iframe width="100%" height="20" scrolling="no" frameborder="no" allow="autoplay; encrypted-media" src="https://w.soundcloud.com/player/?url=https%3A//api.soundcloud.com/tracks/soundcloud%253Atracks%253A2406525546&color=%23ff5500&inverse=false&auto_play=false&show_user=true"></iframe><div style="font-size: 10px; color: #cccccc;line-break: anywhere;word-break: normal;overflow: hidden;white-space: nowrap;text-overflow: ellipsis; font-family: Interstate,Lucida Grande,Lucida Sans Unicode,Lucida Sans,Garuda,Verdana,Tahoma,sans-serif;font-weight: 100;"><a href="https://soundcloud.com/bryantmcgill" title="Bryant McGill" target="_blank" style="color: #cccccc; text-decoration: none;">Bryant McGill</a> · <a href="https://soundcloud.com/bryantmcgill/harms-incurred-while-bringing" title="Harms Incurred While Bringing Preventive Systems Online" target="_blank" style="color: #cccccc; text-decoration: none;">Harms Incurred While Bringing Preventive Systems Online</a></div> ## The Second Error Function in Predictive Security: Endogenous Ground Truth, Legibility-Gated Redress, and Compounding Downstream Destruction About twenty thousand U.S. persons filed inquiries with the DHS Traveler Redress Inquiry Program between December 7, 2021 and September 30, 2023. The [[wiki/Government Accountability Office|Government Accountability Office]] found that 289 of those inquiries concerned the terrorist watchlist. In **88 of the 289, roughly one in three, the person was removed from the list during [[wiki/Watchlist Redress|redress]]**. This is a correction yield among a self-selected redress population, not a watchlist-wide error rate. Nearly two decades earlier the same office reported what happened when screening agencies sent tens of thousands of potential matches to the [[wiki/Terrorist Screening Center|Terrorist Screening Center]] between late 2003 and early 2006. **About half turned out to be [[wiki/Watchlist Misidentification|misidentifications]]**, and the people involved went through intensive questioning, searches, missed flights and trouble at borders. In 2014 a Facebook data scientist and two Cornell collaborators published an intervention in [[wiki/Proceedings of the National Academy of Sciences|PNAS]]. They shifted the News Feeds of **689,003 users** toward positive or negative emotional content, then measured how the users' own emotional expression changed. Cornell's institutional review board treated the university's role as analysis of data it had not collected, so no human-subjects review ever covered the intervention itself. In the year ending March 2024, the United Kingdom's [[wiki/Prevent|Prevent]] programme received **6,922 referrals**. Forty percent came from the education sector, and **512, about seven percent, were adopted as [[wiki/Channel Programme|Channel]] cases**. The government's own Independent Review of Prevent recorded that evaluations of some Prevent-funded projects had found **detrimental effects on participants**. These included reduced support for freedom of expression and, in three evaluated projects, lower trust in police, government and media. The four records are one process at four different moments. A system estimates hidden human states from incomplete observations, ranks people by forecast consequence, and intervenes. It learns while it operates. The people whose lives supply its training signal absorb costs that the system has no built-in way to count. The result can be **compounding downstream destruction across mobility, immigration, employment, education, family, reputation, finances, psychological stability, relationships, and future opportunity**. This paper maps those harms through [[wiki/Security Harms Topology|Security Harms Topology]] and calls the missing measurement instrument the **[[wiki/The Second Error Function|second error function]]**. Every mature protective system already computes a **first error function**: the catastrophes it failed to prevent, the plots it missed, the outbreaks it forecast too late. The second error function counts the harms the system produced while trying to prevent those catastrophes: - people misidentified - errors spread through contaminated edges into associates, places and organizations - states changed by the intervention itself (iatrogenic changes) - estimates the affected person could never contest - data that migrated to purposes other than the one it was collected for - trust that decayed and took future sensing capacity with it The argument here favors prediction and protection. A predictive security architecture reaches decision-grade maturity only when both error functions feed its learning loop. A system that computes only the first becomes an **asymmetric learner**. Observations can be wrong, and edges spread the error. Interventions alter the thing being measured, and the altered state returns as training data. Secondary institutions act on the score. Meanwhile the costs carried by false and marginal positives never flow back to the model as negative feedback. ## **The Austin grammar** The collection's [[wiki/CVE-CVE Convergence|CVE-CVE Convergence]] node states the governing loop: **observe → identify → correlate → reconstruct relationships → infer state → forecast propagation → prioritize → intervene → verify outcome → update the model**. The name collision between [[wiki/Common Vulnerabilities and Exposures|Common Vulnerabilities and Exposures]] and [[wiki/Countering Violent Extremism|Countering Violent Extremism]] is useful because both fields run this sequence on partially observed threats moving through connected systems. Austin supplies the clearest public instantiation of the loop. - **[[wiki/Meyers Lab|Meyers Lab]] modeling and its defense transfer.** Under [[wiki/Lauren Ancel Meyers|Lauren Ancel Meyers]], the lab models susceptibility, exposure, transmission, intervention and uncertainty. In 2018 it used [[wiki/Texas Advanced Computing Center|TACC]] resources to evaluate more than six hundred candidate influenza data streams, and the resulting forecasting methods went to the Defense Threat Reduction Agency's [[wiki/Biosurveillance Ecosystem|Biosurveillance Ecosystem]]. - **The [[wiki/COVID-19|COVID-19]] operational chain.** In March 2020 the [[wiki/UT COVID-19 Modeling Consortium|UT COVID-19 Modeling Consortium]] joined the lab to TACC, [[wiki/Dell Medical School|Dell Medical School]], health systems and [[wiki/Austin Public Health|Austin Public Health]]. Dell clinicians turned transmission models into realistic hospital-capacity and treatment conditions. Researchers studied tracing delays and intervention effectiveness using contact-tracing data that Dell Medical School collected under Austin Public Health authority, the practical form of [[wiki/Contact Tracing as Network Inference|contact tracing as network inference]]. - **The [[wiki/Oden Institute|Oden Institute]] methods foundry.** Oden supplies the underlying mathematics: [[wiki/Inverse Problem|inverse problems]] that reconstruct hidden conditions from fragments, [[wiki/Data Assimilation|data assimilation]] that updates the representation, [[wiki/Uncertainty Quantification|uncertainty quantification]] that sets how seriously a forecast should be taken, and [[wiki/Digital Twin|digital-twin]] logic that compares the model with the thing it models. The documented Meyers and Dell Medical School record lies entirely in public-health forecasting and decision support. Inside that record the loop runs openly: published forecasts, public dashboards, staged alerts and retrospective evaluation. Its security instantiation applies the same grammar to violent mobilization, identity and threat networks, including the [[wiki/Complex Contagion of Violent Extremism|complex-contagion model of violent extremism]]. On the [[wiki/Counterterrorism Predictive Graph Evidence Ladder|evidence ladder]] that instantiation stands at documented capability, documented cross-domain method transfer, documented sponsor mission relevance, and strongly indicated [[wiki/Foreseeable Dual Use|foreseeable dual use]]. Named operational deployment remains unresolved. This paper works in that space. It asks what harms the security instantiation produces while it is being brought online. It answers from the overt programs whose harms reached the public record, because those programs map the whole harm space, including the part that no record documents. ## **Deemed not to be research** The revised Common Rule, adopted by DHS at 6 CFR 46.102, lists four classes of activity that are **deemed not to be research** for purposes of human-subjects regulation. Two of them match the two halves of the CVE-CVE convergence exactly, which gives the convergence a regulatory form documented in [[wiki/Operational Activity and the Common Rule|Operational Activity and the Common Rule]]: - **Clause (l)(2)** covers **public health surveillance activities**: the collection and testing of information or biospecimens conducted, supported, requested, ordered, required or authorized by a public health authority. The contact-tracing and forecasting chain sits here. - **Clause (l)(4)** covers **authorized operational activities, as determined by each agency, in support of intelligence, homeland security, defense, or other national security missions**. The counterterrorism loop sits here. Executive Order 12333 §2.10 separately requires that any [[wiki/Intelligence Community|Intelligence Community]] research on human subjects follow HHS guidelines and document the subject's informed consent. HHS stresses that other laws, regulations and policies continue to govern public-health surveillance even outside the Common Rule. The precise structure is a **governance seam**. A continuous operational loop and a discrete research study fall under different oversight regimes even when their techniques are analytically identical. The World Health Organization's surveillance-ethics guidance describes the same seam from the public-health side. Surveillance is continuous and generally does not pass through the review built for discrete human-subject research, so states need continuous ethical oversight of their own. WHO singles out reuse of surveillance data for security purposes as requiring scrutiny. The seam matters because it is where **subject status** is assigned. A person inside a research protocol is a subject, with consent, a right to withdraw, adverse-event reporting and a named investigator answerable for the design. The same person inside an operational loop is an entity: a record, a node, a score, an encounter. The protections that attach to entity status are real, and the record shows them working: - DHS TRIP removed 88 people from the watchlist. - *Latif* litigation forced revision of redress procedures. - In *FBI v. Fikre*, the Supreme Court held that a plaintiff's challenge survived his removal from the list. - A bipartisan Senate investigation audited fusion-center reporting. - California's state auditor audited the CalGang database. Every one of these mechanisms depends on the **harmed party's awareness of harm**, or on an auditor choosing to look. The Facebook and Cornell case shows the seam operating at scale. The platform ran the intervention, the university analyzed the results, and neither reviewed the intervention as human-subjects research. The subjects learned they had been subjects only because academic publishing norms forced disclosure. PNAS then attached an editorial expression of concern to the paper. That is a documented case of an informational environment being altered, a behavioral state change being measured, and publishable causal knowledge being produced across hundreds of thousands of unwitting people. It surfaced because a journal sat at the end of the pipeline. That yields the refinement at the center of this paper: **redress is [[wiki/Legibility-Gated Redress|legibility-gated]]**. The harms that enter the record are the harms the harmed can see. - **Legible harms generate inquiries.** Denied boarding is a locked gate with a timestamp, so it generates DHS TRIP inquiries. - **Illegible harms generate nothing.** Each of these arrives as ambient misfortune: - an account closed under a separate AML or counter-terrorist-financing de-risking process with no stated reason - a vendor background check that stalls - a friend who was quietly approached as a source - a score that crossed an unseen threshold and changed the priority of a later encounter **The complaint count measures legibility more than it measures harm**, and the record undercounts each harm class in proportion to how hard it is to see. The nearly one-in-three removal result is a **correction yield among U.S.-person watchlist-related DHS TRIP inquiries identified by GAO**, not an estimate of the watchlist-wide error rate. Its denominator is composed of people who experienced enough visible friction to file redress and whose cases GAO identified as watchlist-related. Every less legible harm class goes without even that denominator. ## **Deployment creates an endogenous data-generation loop** A predictive system for violent mobilization faces a severe [[wiki/Base Rate|base-rate]] problem. The event it targets is rare, so labeled positives are scarce and [[wiki/False Positive|false positives]] dominate any low-threshold screen. The Prevent statistics show the triage burden that follows from trying to detect a rare future event upstream. Referral and Channel adoption are deliberately different stages, so the 6,922-to-512 ratio measures a funnel rather than an error rate, but it gives the scale of human attention consumed per adopted case. Builders have a broad set of tools for estimating causal structure: natural experiments, [[wiki/Causal Inference|observational causal inference]], historical data, simulation, synthetic populations, randomized trials and A/B testing. The Oden and Meyers lineage is expert in exactly the simulation and synthetic-population end of that range. The decisive structural fact arrives with deployment. **Once a model decides where attention goes, it decides which data become visible.** Machine-learning research has formalized this in three results: - **[[wiki/Selective Labels|Selective labels]].** Lakkaraju and colleagues showed that outcomes are observed only for cases the decision-maker acted on, so the counterfactual outcomes of cases left alone stay unobserved. - **[[wiki/Performative Prediction|Performative prediction]].** Perdomo and colleagues showed that deploying a prediction changes the distribution it predicts. - **[[wiki/Runaway Predictive Feedback Loop|Runaway feedback loops]].** Ensign, Friedler, Neville, Scheidegger and Venkatasubramanian proved the mechanism for [[wiki/Predictive Policing|predictive policing]]. A model trained on incidents discovered by police sends police to a location. Police find more incidents there because they are looking there. Those incidents become training data, and the model grows steadily more certain the location is dangerous, even when the underlying crime distribution does not justify that confidence. Lum and Isaac gave the practical version: a system trained on historical enforcement ends up **predicting future policing instead of future crime**. The general principle is **[[wiki/Observation-Induced Ground Truth|observation-induced ground truth]]**. Whenever the act of observing changes what gets observed, the training set is endogenous to the security system. A deployed security model can acquire labels through three regimes, and each carries its own harm. - **Proxy labeling.** The system's outputs (referrals, listings, detentions, interdictions) become the labels it is later validated against, so the model partly manufactures its own ground truth. - **Deferred intervention.** A forecast is allowed to mature so its outcome can be observed. This is the control-arm structure, and the people in the control arm bear its risk. - **Perturbation.** A stimulus (an exposure, a contact, a narrative, an opportunity, a pressure) is introduced and the response measured. Perturbation is established as a capability, as the emotional-contagion study shows, and as a case practice under prosecutorial authority, as the informant record below shows. Whether any program uses perturbation as a systematic calibration method is unresolved. The capability, the precedent and the calibration problem are all established. Under the Maximum-Implementation Principle that governs this collection, builders who hold both a perturbation capability and a calibration problem understand how the two relate. ## **The harm topology** ### **Identification: entity-resolution error becomes inherited friction** [[wiki/Watchlist Misidentification|Watchlist misidentification]] contains two different root failures. **[[wiki/Identity Collision|Identity collision]]** occurs when a person who is not listed is provisionally matched to somebody else's watchlist identity because names or other identifiers resemble one another. **[[wiki/Erroneous Watchlist Inclusion|Erroneous inclusion]]** occurs when the system has found the correct person but the threat state attached to that person is mistaken, unsupported, incomplete, obsolete, or no longer warranted. The first is a wrong-entity problem; the second is a wrong-state problem. They produce different harms and require different remedies. The public record quantifies both. GAO reported in 2006 that about half of the tens of thousands of ambiguous potential matches escalated to the [[wiki/Terrorist Screening Center|Terrorist Screening Center]] between December 2003 and January 2006 were misidentifications. This was not half of everyone screened; it was half of the potential matches that frontline systems could not resolve. DOJ's Inspector General reported in 2007 that 38 percent of 105 records tested after routine quality review still contained errors or inconsistencies. Among 388 closed redress complaints, 52 involved people who were not the listed identity, while 97 positive-match records were modified and 76 were removed. GAO's 2025 review found that 88 of 289 U.S.-person watchlist-related DHS TRIP inquiries ended in removal during redress. The matching system must balance evasion risk against collision burden. A narrow rule misses aliases, misspellings, transliterations, partial dates of birth, and other adversarial or degraded identifiers. A broad rule catches more variations while increasing false matches. GAO documented that widening Secure Flight's date-of-birth range would catch additional simulated records but, according to TSA, generate an unacceptable number of false positives. The engineering objective is therefore not zero uncertainty at the first screen; it is fast, accurate, and minimally harmful resolution after uncertainty appears. The catastrophic form is a **[[wiki/Distributed Erroneous State|distributed erroneous state]]**. The FBI says terrorism-watchlist information supports visa and passport screening, border entry, aviation screening, immigration screening, military-base access, FBI investigations, and federal, state, local, tribal, and territorial law enforcement, with some information exported through NCIC. Once the root state is wrong, each receiving system can act correctly on the record and still produce an externally wrong result. The error becomes a property of the network rather than one database row. [[wiki/Rahinah Ibrahim|Rahinah Ibrahim]] is the canonical documented case. An FBI agent misunderstood a nomination form and checked boxes contrary to its instructions, placing her on the No Fly List and IBIS. The state propagated into arrest and detention at San Francisco International Airport, missed travel, watchlist exports, student-visa revocation, later visa denials, inability to return to the United States, and years of litigation. The government eventually conceded that she posed no threat to national security and should never have been placed on the No Fly List. [[wiki/Ibrahim v. Department of Homeland Security|Her case]] demonstrates how one input error can become a distributed institutional reality. The affected person then adapts. They document encounters, change routes, avoid travel, contact agencies, file DHS TRIP and FOIA requests, hire counsel, ask associates what occurred, and try to reconstruct the invisible cause. Those reasonable adaptations create new records and behaviors. If later interpretation begins from the original threat state, the system's reaction changes the person, and the person's changed behavior returns as apparent evidence. That is [[wiki/Identity-Rooted Error Amplification|identity-rooted error amplification]]. The endpoint is [[wiki/Life-Trajectory Harm|life-trajectory harm]]: **compounding downstream destruction across mobility, immigration, employment, education, family, reputation, finances, psychological stability, relationships, and future opportunity**. Each institution sees one fragment. The person lives the accumulation. The system can distribute harm more efficiently than it distributes awareness of harm. Correction is itself distributed. The FBI's 2024 transparency document says that when a record is modified or removed, the [[wiki/Terrorist Screening Center|Terrorist Screening Center]] verifies that the change carries into the receiving screening systems. The 2007 DOJ audit found an average 67-day TSC redress review and a case in which CBP took more than 130 days to make a requested downstream change. [[wiki/Correction Latency|Correction latency]] is therefore part of the harm, not merely an administrative delay. The architecture has also built an anti-collision identity layer. The FBI's Terrorist Screening Records System historically maintained a category for **misidentified persons** and additional attributes used to distinguish them from the actual listed identity. DHS TRIP assigns a Redress Control Number that travelers can provide in future reservations to help suppress repeat collisions. This is a [[wiki/Negative Identity Assertion|negative identity assertion]]: *this person is not that watchlisted person*. The cure for repeated misidentification requires the system to remember the misidentification. [[wiki/Identity Fusion|Identity fusion]] and its inverse, fission, generalize the primitive. Two people merged into one node inherit each other's exposure histories. A person split across records leaves a fragment carrying a state estimate that belongs to no one. The resulting burden should be measured as [[wiki/Cumulative Downstream Burden|cumulative downstream burden]]: encounters, agencies reached, derivative decisions, correction time, lagging copies, associate records, lost opportunity, and residual effects per corrected root error. ### **Relationship reconstruction: guilt by edge and graph error propagation** [[wiki/Network Epidemiology|Network epidemiology]] treats exposure probabilistically. That is appropriate for a respiratory pathogen, where proximity is the transmission medium. Carried over into models of ideological contagion, **adjacency becomes a scoring variable**. The second-degree ring around a node (family, congregants, clients, students, a therapist's caseload, a shared gym or server) absorbs part of the node's estimated risk. The NYPD's Demographics Unit is the overt, community-scale form. It mapped Muslim communities and their institutions, from mosques to bookstores to restaurants, and sent paid informants into mosques, student associations and community events. In a 2012 deposition, the Intelligence Division's commanding officer testified that the unit's work had never generated a lead or triggered a terrorism case. The program ended through the *Raza* settlement in 2016 and the *Hassan* settlement in 2018. Its record shows adjacency scoring operating at the scale of a religious community, with its harm recorded because litigation compelled testimony. California's state auditor documented how such a graph degrades when its nodes are weak. The CalGang criminal-intelligence database held questionable entries that agencies could not substantiate, oversight was inadequate to protect privacy, and 42 of the people in it had been entered before they were a year old. [[wiki/Terrorist Financing|Counter-terrorist-financing]] compliance supplies the institutional version. GAO documented **de-risking**: banks restrict or end services to money transmitters and nonprofits working in regions seen as high-risk for money laundering or terrorist financing, which produces account closures and delayed or denied transfers with no allegation against the affected customer. The engineering primitive is **[[wiki/Graph Error Propagation|graph error propagation]]**. The dangerous failure is more than one bad classification. Once a node is wrong, its edges carry the error into associates, places, organizations and every downstream system that ingests the graph. That is [[wiki/Error Propagation|error propagation]] in its most consequential form. ### **State inference: the instrument's provenance and the uncontestable estimate** The Independent Review of Prevent acknowledged three things about Prevent's [[wiki/Vulnerability Assessment|Vulnerability Assessment Framework]]: it comes from the ERG22+ factors, which were developed for convicted extremist offenders; the evidence base came from a small sample; and the evidence behind the framework has been criticized. The National Institute of Justice places ERG22+, VERA-2 and related tools as structured aids to professional judgment that stop short of actuarial-grade prediction of extremist recidivism. They are decision-support instruments, and their outputs deserve the uncertainty such instruments carry. ERG22+ includes a factor for the **need to redress injustice and express grievance**. Instruments weighted toward grievance create a specific hazard. A person protesting an erroneous state estimate may display the very features the instrument weights, so contestation itself can register as a risk feature. Whether redress activity enters any scoring pipeline is unresolved. That grievance factors are weighted at the instrument level is established. The adjacent primitive is **[[wiki/Redress Opacity|redress opacity]]**: a subject cannot correct an estimate whose content and provenance are withheld. - In the *Latif* No Fly List litigation, plaintiffs challenged a system in which the government generally would neither confirm listing status nor disclose enough to contest it. The district court found serious [[wiki/Due Process|due-process]] deficiencies, and the government revised its procedures. - In *FBI v. Fikre* (2024), a unanimous [[wiki/Supreme Court of the United States|Supreme Court]] held that removing a plaintiff from the No Fly List during litigation did not automatically moot his challenge. The opinion recounts, as an allegation not yet proven at that stage, that agents told Fikre continued listing could be avoided if he became an [[wiki/Informant|informant]]. Put the two together and the latent state is simultaneously consequential, hidden and potentially usable as leverage. That combination is the highest-risk configuration in the topology. ### **Forecast and prioritization: runaway confidence and resource diversion** Runaway feedback is the forecast-stage harm. The model's confidence rises with the enforcement it directs, independent of the underlying distribution. Its companion is **resource diversion**, which joins the second error function to the first. TSA spent roughly $900 million on the SPOT [[wiki/Behavior Detection|behavior-detection]] program while GAO repeatedly found insufficient scientific evidence that its indicators could reliably identify aviation threats. In 2017, GAO found valid supporting evidence for only **8 of 36 revised indicators**. The travelers screened under the program were screened by an instrument that was never validated. The [[wiki/Fusion Center|fusion-center]] layer shows the same cost at the telemetry level: - A bipartisan investigation by the Senate Permanent Subcommittee on Investigations concluded in 2012 that DHS-linked fusion-center reporting was of uneven quality, often shoddy, frequently unrelated to terrorism, and in some instances a danger to privacy and civil liberties. - GAO recorded a senior Justice Department official warning that poor fusion-center handling could harm individuals, spread inaccurate data, erode public confidence and create liability. - DHS itself directs that [[wiki/Suspicious Activity Reporting|suspicious-activity reporting]] rest on behavior, never on race, religion or appearance, because much apparently suspicious activity is innocent. The primitive is that **low-specificity telemetry has downstream cost**. A suspicious-activity report becomes an edge, an attribute, a confidence update and a persistent record across several systems. In cybersecurity this is alert fatigue. In counterterrorism, analyst attention spent on false and low-value signals is attention withheld from real threats, so the second error function's costs return as first-error-function risk. ### **Intervention: iatrogenic security intervention and the perturbation record** The Independent Review of Prevent establishes the phenomenon at the center of this paper: **counter-radicalization interventions can produce counterproductive state changes**, the pattern named [[wiki/Iatrogenic Security Intervention|iatrogenic security intervention]]. The review recorded: - Evaluations of some Prevent-funded programs found detrimental effects on participants, including lower support for freedom of expression and, in three projects, lower trust in police, government and media. - Higher-education professionals reported that Prevent could chill discussion, erode trust between students and staff, and make some people less willing to take part because they were unsure how their personal information would be used. The CVE literature treats institutional distrust, grievance and social isolation as risk accelerants. **An intervention that lowers trust moves its participants toward the profile it was meant to reduce**, and a model that scores the post-intervention state will read the change as confirmation of its forecast. This is the iatrogenic loop, and it is the most dangerous harm class because it validates itself: measured accuracy rises in step with the harm. The informant record is the perturbation method working under prosecutorial authority. Human Rights Watch's 2014 investigation, *Illusion of Justice*, documented federal terrorism cases in which informants played central roles in developing plots, supplied resources, and targeted particularly vulnerable people. In the Newburgh Four case the federal judge said the government had in effect supplied the crime and the means and removed the obstacles. These are documented cases and HRW's findings about them. They establish intervention-induced state change as a practice. Whether it functions as a general calibration method is unresolved. ### **Verification and update: purpose migration and the trust externality** The update stage is where data crosses authorities. Two cases show the pattern: - **Singapore.** TraceTogether data was collected for contact tracing and COVID-19 control. In January 2021 the government clarified that police had legal authority to obtain it for criminal investigations, and later legislation narrowed permissible criminal use to a defined set of serious offences. - **Germany.** In Mainz, prosecutors, police and the local health authority used Luca contact-tracing records in a criminal investigation despite rules limiting those records to infection control, and the Rhineland-Palatinate data-protection authority opened proceedings. WHO's guidance now holds that identifiable surveillance data generally should not be shared for unrelated law-enforcement, national-security or social-benefit purposes without compelling justification and legal process. The primitive is **[[wiki/Surveillance Purpose Migration|surveillance purpose migration]]**, the operational form of [[wiki/Function Creep|function creep]]. The Austin chain shows why the relational asset matters. Contact-tracing data collected by Dell Medical School under Austin Public Health authority, used there for public-health effectiveness research, is exactly the exposure graph a threat-network model wants. The harm vector is the portability of such datasets across authorities, and the governance seam determines whether that portability is ever reviewed. Migration carries a measurable security cost, the **[[wiki/Surveillance Trust Externality|surveillance trust externality]]**: - A U.S. survey of 6,284 adults found generally low approval for COVID-19 digital-data uses and concluded that public-health authorities needed trust-building strategies to secure adoption of [[wiki/Digital Contact Tracing|digital contact tracing]]. - A longitudinal British study found that privacy concerns and beliefs about third-party access were associated with not using the app, that trust in government predicted later adoption, and that a perceived lack of transparent evidence contributed to people abandoning the NHS app. Misuse therefore degrades the sensors themselves. The population withholds data, disables apps, avoids contact with the system or supplies less reliable information. **An abuse today lowers the resolution of the next pandemic or security event**, which is a second way the second error function returns as first-error-function risk. ### **Topology at a glance** | Loop stage | Harm primitive | Documented anchor | Tier | | --- | --- | --- | --- | | Identify | [[wiki/Watchlist Misidentification\|Watchlist misidentification]]: [[wiki/Identity Collision\|identity collision]] or [[wiki/Erroneous Watchlist Inclusion\|erroneous inclusion]] becomes a [[wiki/Distributed Erroneous State\|distributed erroneous state]] | GAO 2006; DOJ OIG 2007; *Ibrahim*; GAO 2025 and 2026 | Established | | Correlate / reconstruct | Guilt by edge; [[wiki/Graph Error Propagation\|graph error propagation]] | NYPD Demographics Unit; CalGang audit; GAO on de-risking | Established | | Infer state | Instrument provenance; [[wiki/Redress Opacity\|redress opacity]]; grievance weighting | Independent Review of Prevent on the Vulnerability Assessment Framework; NIJ; *Latif*; *Fikre*; ERG22+ grievance factor | Established (redress entering scoring: unresolved) | | Forecast / prioritize | [[wiki/Runaway Predictive Feedback Loop\|Runaway confidence]]; resource diversion; low-specificity telemetry | Ensign et al.; Lum and Isaac; GAO on SPOT; Senate investigation of fusion centers | Established | | Intervene | [[wiki/Iatrogenic Security Intervention\|Iatrogenic state change]]; intervention-induced state change | Independent Review of Prevent evaluations; HRW *Illusion of Justice* | Established (systematic calibration use: unresolved) | | Verify / update | [[wiki/Surveillance Purpose Migration\|Purpose migration]]; [[wiki/Surveillance Trust Externality\|trust externality]] and sensor degradation | TraceTogether; Luca/Mainz; WHO; JAMA Network Open; BMJ Open | Established | ## **The mirror ledger** The [[wiki/Meyers Lab|Meyers Lab]] record includes a 2025 PNAS estimate that influenza vaccination averted **69,886 hospitalizations** during the 2022–2023 U.S. season. It is a model-based counterfactual with uncertainty: no one can name the people who were spared, and the number carries scientific authority anyway because the modeling apparatus earns it. Prevention science grounds its legitimacy in counterfactual accounting of its benefits. Counterterrorism does the same when it cites plots disrupted. The machinery that estimates averted harm can estimate inflicted harm with equal rigor, and the public record already contains a first measurement for each term of the second error function: - **Collision frequency and resolution burden:** potential matches, negative resolutions, repeat encounters after clearance, and analyst time required to distinguish one identity from another. - **Erroneous-inclusion correction yield:** the share of a defined redress cohort whose record is modified or removed, reported with the cohort-selection rule rather than promoted into a population-wide error rate. - **Correction latency:** time from complaint or authoritative correction to closure and to propagation through every receiving screening system. - **Cumulative downstream burden:** encounters, agencies reached, derivative decisions, associated people recorded, lost travel, work and educational opportunities, financial cost, behavior change, and residual effects per corrected root error. - **Listing persistence after the basis lapses**, which GAO has named as a distinct mechanism. - **Collateral edge burden:** how many second-degree nodes had their scores moved by a node later corrected. The CalGang audit is its closest public analog. - **The iatrogenic delta:** the change in trust, isolation and grievance measures among intervention participants relative to a comparison group, which Prevent's own evaluations already measure. - **Purpose-migration events per dataset**, which Singapore and Mainz have already produced. - **The trust elasticity of sensing:** how far participation falls after a disclosed migration, which the British longitudinal work begins to quantify. - **Resource diversion:** analyst hours and program dollars per validated lead, which the SPOT audits and the NYPD testimony have partly computed. **The second error function is buildable from instruments that already exist.** What it needs is a place in the learning loop, a mandate to be computed continuously alongside the first, and a governance regime that covers the operational seam where subject status is currently assigned away. ## **Forensic signatures** Harms that are invisible one person at a time become visible in aggregate. That is the same statistical position the builders themselves work from. The residue shows up in the following places: - **Correlated financial and credential events.** Clusters of de-risking closures, credential denials and screening selections that track network position more closely than individual conduct. - **Isolation around flagged nodes.** Excess social isolation in the second-degree rings of flagged nodes. - **Post-intervention confirmation spikes.** A jump in predicted-state confirmation right after an intervention is the iatrogenic signature. - **Agency (l)(4) determinations.** These are administrative decisions that should exist as records, and FOIA can reach them. - **University non-research determinations.** At institutions holding defense, intelligence or homeland-security research relationships, these mark where a human-facing project was placed outside review. - **Redress statistics over time.** Removal rates across successive reporting periods measure whether listing error is shrinking or being redistributed into less visible harm classes. ## **Epistemic tier ledger** - **Established:** - the Common Rule's (l)(2) and (l)(4) exclusions for public-health surveillance and operational intelligence activity, and agency-level determination of operational activity - EO 12333 §2.10 and HHS's statement that other governance continues to apply - WHO's continuous-oversight and data-reuse guidance - GAO's watchlist misidentification and redress findings (2006, 2025) - *Latif* and *Fikre* - the Prevent, NYPD Demographics Unit, SPOT, informant-sting and Facebook/PNAS records - the Ensign et al. and Lum and Isaac feedback-loop results; the selective-labels and performative-prediction formalisms - the Independent Review of Prevent's findings on detrimental effects, trust, chilling effects and the Vulnerability Assessment Framework's provenance - Prevent's 2023–24 referral and Channel figures - NIJ's characterization of structured risk instruments - the Senate findings on fusion centers and GAO's fusion-center record - the CalGang audit - GAO's de-risking findings - the TraceTogether and Luca data migrations - the U.S. and British trust studies - the Meyers → BSVE methods transfer and the Austin COVID-19 operational chain - **Strongly indicated:** - that deployed predictive CVE systems validate largely through proxy labels and endogenous, attention-dependent data, a consequence of base rates and selective labels - that grievance-weighted risk instruments can turn contestation of an estimate into a risk feature - that harms missing from the record are concentrated in the least legible harm classes - **Plausible:** - operational intervention cohorts structured like trials - perturbation used deliberately as calibration inside security programs - entity-resolution fusion producing persistent inherited friction on a scale well beyond what redress statistics capture - **Unresolved:** - which named programs have used which label-generating regime, on whom, and when - whether redress-seeking behavior enters any scoring pipeline - whether Austin-origin relational health data has entered a threat-model pipeline The documented Meyers and Dell Medical School record sits entirely in public-health forecasting and decision support. The harm space mapped here belongs to the security instantiation of the same loop, which the evidence ladder places at strongly indicated foreseeable dual use. ## **Relationships** - **master collection:** [[collections/Terrorism, Counterterrorism, and the Intelligence Environment|Terrorism, Counterterrorism, and the Intelligence Environment]]. - **central concepts:** [[wiki/The Second Error Function|The Second Error Function]] and [[wiki/Security Harms Topology|Security Harms Topology]]. - **governing loop:** [[wiki/CVE-CVE Convergence|CVE-CVE Convergence]], [[wiki/Human CVE|Human CVE]], [[wiki/Latent State Estimation|Latent State Estimation]], and [[wiki/Intervention Point|Intervention Point]]. - **Austin demonstration:** [[wiki/Meyers Lab|Meyers Lab]], [[wiki/Dell Medical School|Dell Medical School]], [[wiki/Austin Public Health|Austin Public Health]], [[wiki/Texas Advanced Computing Center|TACC]], [[wiki/Oden Institute|Oden Institute]], [[research/The Austin Executable Loop|The Austin Executable Loop]], and [[research/The Austin Surveillance Field|The Austin Surveillance Field]]. - **contagion bridges:** [[wiki/Contact Tracing as Network Inference|Contact Tracing as Network Inference]] and [[wiki/Complex Contagion of Violent Extremism|Complex Contagion of Violent Extremism]]. - **identification layer:** [[wiki/Watchlist Misidentification|Watchlist Misidentification]], [[wiki/Identity Collision|Identity Collision]], [[wiki/Erroneous Watchlist Inclusion|Erroneous Watchlist Inclusion]], [[wiki/Negative Identity Assertion|Negative Identity Assertion]], [[wiki/Watchlisting|Watchlisting]], [[wiki/Watchlist Redress|Watchlist Redress]], [[wiki/Threat Screening Center|Threat Screening Center]], [[wiki/Terrorist Screening Center|Terrorist Screening Center]], [[wiki/Terrorist Screening Database|Terrorist Screening Database]], [[wiki/National Crime Information Center|National Crime Information Center]], [[wiki/Entity Resolution|Entity Resolution]], and [[wiki/Identity Fusion|Identity Fusion]]. - **distributed-harm layer:** [[wiki/Distributed Erroneous State|Distributed Erroneous State]], [[wiki/Identity-Rooted Error Amplification|Identity-Rooted Error Amplification]], [[wiki/Correction Latency|Correction Latency]], [[wiki/Cumulative Downstream Burden|Cumulative Downstream Burden]], and [[wiki/Life-Trajectory Harm|Life-Trajectory Harm]]. - **canonical case:** [[wiki/Rahinah Ibrahim|Rahinah Ibrahim]] and [[wiki/Ibrahim v. Department of Homeland Security|Ibrahim v. Department of Homeland Security]]. - **graph layer:** [[wiki/Graph Error Propagation|Graph Error Propagation]], [[wiki/Network Epidemiology|Network Epidemiology]], [[wiki/Error Propagation|Error Propagation]], and [[wiki/Graph-to-Collection Feedback Loop|Graph-to-Collection Feedback Loop]]. - **learning layer:** [[wiki/Observation-Induced Ground Truth|Observation-Induced Ground Truth]], [[wiki/Selective Labels|Selective Labels]], and [[wiki/Runaway Predictive Feedback Loop|Runaway Predictive Feedback Loop]]. - **prevention layer:** [[wiki/Iatrogenic Security Intervention|Iatrogenic Security Intervention]], [[wiki/Prevent|Prevent]], [[wiki/Channel Programme|Channel Programme]], [[wiki/Vulnerability Assessment|Vulnerability Assessment]], and [[wiki/Counterterrorism Collection - Public Health and Violence Prevention|Public Health and Violence Prevention]]. - **forecast layer:** [[wiki/Predictive Policing|Predictive Policing]], [[wiki/Hotspot Analysis|Hotspot Analysis]], [[wiki/Behavior Detection|Behavior Detection]], [[wiki/Fusion Center Oversight|Fusion Center Oversight]], and [[wiki/Suspicious Activity Reporting|Suspicious Activity Reporting]]. - **governance layer:** [[wiki/Operational Activity and the Common Rule|Operational Activity and the Common Rule]], [[wiki/Legibility-Gated Redress|Legibility-Gated Redress]], [[wiki/Redress Opacity|Redress Opacity]], [[wiki/Surveillance Purpose Migration|Surveillance Purpose Migration]], [[wiki/Surveillance Trust Externality|Surveillance Trust Externality]], [[wiki/Due Process|Due Process]], [[wiki/Disclosure Asymmetry|Disclosure Asymmetry]], [[wiki/Function Creep|Function Creep]], [[wiki/Institutional Trust|Institutional Trust]], [[wiki/Structural Redress|Structural Redress]], and [[wiki/Counterterrorism Collection - Privacy and Civil Liberties|Privacy and Civil Liberties]]. - **evidence discipline:** [[wiki/Counterterrorism Predictive Graph Evidence Ladder|Counterterrorism Predictive Graph Evidence Ladder]] and [[wiki/Foreseeable Dual Use|Foreseeable Dual Use]]. --- [[about/About Bryant McGill|Bryant McGill]] is a Wall Street Journal and USA Today bestselling author, systems architect, technologist, and strategic advisor, as well as a Congressionally Recognized Ambassador of Goodwill and United Nations–appointed Global Champion. His work spans naval intelligence systems, computational linguistics, artificial intelligence, digital transformation, and civilizational governance architecture. His forward analysis on U.S.–Israel Pax Silica frameworks has appeared in Jewish/Jerusalem News Syndicate (JNS). --- ## **Sources** - [45 CFR 46.102 — Definitions, eCFR](https://www.ecfr.gov/current/title-45/subtitle-A/subchapter-A/part-46/subpart-A/section-46.102) - [6 CFR 46.102 — Definitions, DHS Common Rule](https://ecfr.io/Title-6/Section-46.102) - [HHS OHRP — Draft Guidance on Activities Deemed Not to Be Research: Public Health Surveillance](https://www.hhs.gov/ohrp/regulations-and-policy/requests-for-comments/draft-guidance-activities-deemed-not-be-research-public-health-surveillance/index.html) - [Executive Order 12333 — United States Intelligence Activities, National Archives](https://www.archives.gov/federal-register/codification/executive-order/12333.html) - [WHO Guidelines on Ethical Issues in Public Health Surveillance (2017)](https://iris.who.int/bitstream/handle/10665/255721/9789241512657-eng.pdf) - [WHO — Q&A: Ethics in Public Health Surveillance](https://www.who.int/news-room/questions-and-answers/item/q-a-ethics-in-public-health-surveillance) - [GAO-25-108349 — Terrorist Watchlist: Nomination and Redress Processes for U.S. Persons (2025-08-14)](https://www.gao.gov/products/gao-25-108349) - [GAO — Terrorist Watch List Screening: Efforts to Help Reduce Adverse Effects on the Public (2006)](https://www.gao.gov/assets/a251884.html) - [DOJ Office of the Inspector General — Follow-up Audit of the Terrorist Screening Center, Audit Report 07-41 (2007)](https://oig.justice.gov/archives/reports/FBI/a0741/exec.htm) - [GAO-09-292 — Aviation Security: Secure Flight Implementation and Matching Performance (2009)](https://www.gao.gov/assets/a289639.html) - [FBI — Terrorist Screening Records System, 72 FR 47073 (2007)](https://www.fbi.gov/how-we-can-help-you/more-fbi-services-and-information/freedom-of-information-privacy-act/fbi-privacy-act-systems/72-fr-47073) - [FBI — Overview of the U.S. Government's Terrorist Watchlisting Process and Procedures (April 2024)](https://www.fbi.gov/file-repository/terrorist-watchlisting-transparency-document-april-2024-050224.pdf) - [FBI — Threat Screening Center](https://www.fbi.gov/investigate/terrorism/tsc) (accessed 2026-09-23) - [DHS TRIP — Frequently Asked Questions](https://trip.dhs.gov/s/faq-page?language=en_US) (accessed 2026-09-23) - [GAO-26-108650 — Terrorist Watchlist: FBI Should Improve Outreach Efforts to Nonfederal Users (2026-01-12)](https://www.gao.gov/products/gao-26-108650) - [*Ibrahim v. Department of Homeland Security*, 912 F.3d 1147 (9th Cir. 2019)](https://cdn.ca9.uscourts.gov/datastore/opinions/2019/01/02/14-16161.pdf) - [Latif v. Holder, U.S. Court of Appeals for the Ninth Circuit (2012)](https://cdn.ca9.uscourts.gov/datastore/opinions/2012/07/26/11-35407.pdf) - [FBI v. Fikre, Supreme Court of the United States (2024)](https://www.supremecourt.gov/opinions/23pdf/22-1178_p8k0.pdf) - [Ensign et al. — Runaway Feedback Loops in Predictive Policing, PMLR 81 (2018)](https://proceedings.mlr.press/v81/ensign18a.html) - [Lum and Isaac — To Predict and Serve?, Significance (2016)](https://rss.onlinelibrary.wiley.com/doi/abs/10.1111/j.1740-9713.2016.00960.x) - [Lakkaraju et al. — The Selective Labels Problem, KDD (2017)](https://www.cs.cornell.edu/home/kleinber/kdd17-selective.pdf) - [Perdomo et al. — Performative Prediction, ICML (2020)](https://proceedings.mlr.press/v119/perdomo20a.html) - [Independent Review of Prevent's Report and Government Response, GOV.UK (2023)](https://www.gov.uk/government/publications/independent-review-of-prevents-report-and-government-response/independent-review-of-prevent-accessible) - [Individuals Referred to and Supported Through the Prevent Programme, April 2023 to March 2024, GOV.UK](https://www.gov.uk/government/statistics/individuals-referred-to-prevent-to-march-2024/individuals-referred-to-and-supported-through-the-prevent-programme-april-2023-to-march-2024) - [Lessons for Prevent, GOV.UK](https://www.gov.uk/government/publications/lessons-for-prevent/lessons-for-prevent-accessible) - [Ministry of Justice — The Extremism Risk Guidance 22+: An Exploratory Psychometric Analysis](https://assets.publishing.service.gov.uk/media/641c2b8f32a8e0000cfa9288/extremism-risk-guidance-22+.pdf) - [National Institute of Justice — Research and Practitioner Perspectives on the Rehabilitation and Reintegration of Violent Extremists](https://nij.ojp.gov/topics/articles/research-and-practitioner-perspectives-rehabilitation-and-reintegration-violent) - [GAO-14-159 — Aviation Security: TSA Should Limit Future Funding for Behavior Detection Activities (2013)](https://www.gao.gov/products/gao-14-159) - [GAO-17-608R — TSA Does Not Have Valid Evidence Supporting Most of the Revised Behavioral Indicators (2017)](https://www.gao.gov/products/gao-17-608r) - [Senate Permanent Subcommittee on Investigations — Investigative Report Criticizes Counterterrorism Reporting at Fusion Centers (2012)](https://www.hsgac.senate.gov/subcommittees/investigations/rep/investigative-report-criticizes-counterterrorism-reporting-waste-at-state-local-intelligence-fusion-centers/) - [GAO — Fusion Center Information Sharing and Privacy Protections](https://www.gao.gov/assets/a310273.html) - [DHS — What Is Suspicious Activity?](https://www.dhs.gov/see-something-say-something/what-suspicious-activity) - [GAO-18-642T — Bank Secrecy Act: Derisking along the Southwest Border and Money Transmitter Account Closures](https://www.gao.gov/products/gao-18-642t) - [California State Auditor — The CalGang Criminal Intelligence System, Report 2015-130](https://information.auditor.ca.gov/reports/2015-130/summary.html) - [Associated Press — NYPD: Muslim Spying Led to No Leads, Terror Cases (2012-08-21)](http://archive.boston.com/news/nation/washington/articles/2012/08/21/nypd_muslim_spying_led_to_no_leads_terror_cases/) - [ACLU — Raza v. City of New York](https://www.aclu.org/cases/raza-v-city-new-york-legal-challenge-nypd-muslim-surveillance-program) - [ACLU — Landmark Settlement in Challenge to NYPD Surveillance of New York Muslims (2016)](https://www.aclu.org/news/national-security/landmark-settlement-challenge-nypd-surveillance-new) - [Center for Constitutional Rights — Settlement Reached in NYPD Muslim Surveillance Lawsuit (Hassan)](https://ccrjustice.org/home/press-center/press-releases/settlement-reached-nypd-muslim-surveillance-lawsuit) - [Human Rights Watch — Illusion of Justice: Human Rights Abuses in US Terrorism Prosecutions (2014)](https://www.hrw.org/report/2014/07/21/illusion-justice/human-rights-abuses-us-terrorism-prosecutions) - [Kramer, Guillory, and Hancock — Experimental Evidence of Massive-Scale Emotional Contagion Through Social Networks, PNAS (2014)](https://www.pnas.org/doi/10.1073/pnas.1320040111) - [Cornell Chronicle — News Feed: "Emotional Contagion" Sweeps Facebook (2014)](https://news.cornell.edu/stories/2014/06/news-feed-emotional-contagion-sweeps-facebook) - [Cornell eCommons — Research Administration Newsletter on the Facebook Study Review (2014)](https://ecommons.cornell.edu/bitstream/handle/1813/41361/2014_SeptemberOctober.pdf) - [Singapore Ministry of Digital Development and Information — Factsheet on the TraceTogether Programme](https://www.mddi.gov.sg/newsroom/factsheet-tracetogether-programme/) - [Datenschutz Rheinland-Pfalz — Proceedings on the Use of Luca App Contact Data by Prosecutors](https://www.datenschutz.rlp.de/service/aktuelles/detail/nach-erhebung-und-nutzung-von-kontaktdaten-aus-der-luca-app-durch-die-staatsanwaltschaft-datenschu) - [JAMA Network Open — Consumer Views on Using Digital Data for COVID-19 Control (2021)](https://jamanetwork.com/journals/jamanetworkopen/fullarticle/2779953) - [BMJ Open — Longitudinal Study of NHS COVID-19 App Adoption and Abandonment (2022)](https://bmjopen.bmj.com/content/12/1/e053327) - [Bi, Meyers, et al. — Estimated Impact of 2022–2023 Influenza Vaccines on Annual Hospital Burden in the United States, PNAS (2025)](https://pmc.ncbi.nlm.nih.gov/articles/PMC12646225/) - [Flu Season Forecasts Could Be More Accurate with Access to Health Care Companies' Data, UT Austin (2018)](https://news.utexas.edu/2018/09/19/this-data-source-could-enable-better-flu-forecasts/) **As of:** 2026-09-23