# XDO: Executable Markdown and Portable Workflows XDO—also described in the surviving work as **Do**, **DoLang**, `xdo`, and through its principal interpreter `x_do.php`—was an attempt to make a readable document carry not only content, but a controlled program for transforming content and data. The aim was not to invent another general-purpose programming language. It was to create an **intermediate semantic language for executable documents**: a small, readable instruction vocabulary placed between human intention and the larger machinery of a content-management system. A writer or developer could describe actions inside Markdown, operate on named blocks and variables, load structured data, transform it, combine it with templates, select an output representation, and place the result into a target. The document could remain readable as prose while also serving as data, template, workflow, and application specification. In contemporary terms, XDO sits at the intersection of [[wiki/Executable Markdown|executable Markdown]], a [[wiki/Domain-Specific Language|domain-specific language]], a transformation pipeline, and a portable workflow engine. It is an important precursor to modern AI-assisted authoring because it separates natural-language intention from the deterministic, permissioned host that performs the work. > [!summary] Canonical description > **DoLang is a compact, extensible domain-specific language embedded in Markdown and developed within Bryant McGill's CMS architecture to turn documents into controlled computational surfaces. It parses executable regions into a hierarchical program → set → command representation, maintains interpreter-managed scoped state, supports contextual iteration, buffers, structured data, persistence, templates, output projections, and multi-pass evaluation, and dynamically maps DSL verbs onto conventionally named `x_do_*` handlers. XApps and XLib form the application and library boundary beneath the language. The intended pipeline moves from human or natural-language intent into constrained Do instructions, through deterministic execution against runtime state, and back into Markdown and CMS output.** --- ## Why it existed The design objective was to reduce the distance between **authorial intent** and **computational action**. A conventional system separates the document from the program operating on it: ```text content.md application.php ``` DoLang permitted a controlled overlap: ```text document.md ├── prose ├── semantic structure ├── embedded data ├── executable instructions ├── templates and output projections └── generated content ``` The desired operating surface was the document itself. Markdown remained readable as Markdown, while selected regions could cross the membrane from representation into execution. This is the meaning of **living documents** in the XDO architecture. They are not documents containing arbitrary scripts. They are documents in which selected semantic intentions can be executed by a bounded interpreter operating against explicit state, named targets, and application capabilities. ## What this architecture would be called now - **Instructions live beside prose in Markdown.** Modern translation: [[wiki/Executable Markdown|executable Markdown]], [[wiki/Document-Centric Computing|document-centric computing]], and executable documentation. - **A small Do vocabulary stands between intention and code.** Modern translation: a [[wiki/Domain-Specific Language|domain-specific language]] and [[wiki/Human-Readable Intermediate Representation|human-readable intermediate representation]]. - **Natural language is translated into constrained Do instructions.** Modern translation: semantic parsing, [[wiki/Natural Language to DSL Compilation|natural-language-to-DSL compilation]], and schema-constrained planning. - **Program → set → command parsing.** Modern translation: an abstract syntax tree and hierarchical execution model. - **Open, parse, execute, and close passes.** Modern translation: compiler/interpreter passes and [[wiki/Multi-Pass Processing|multi-pass processing]]. - **`x_do_*` verbs are resolved dynamically.** Modern translation: a [[wiki/Capability Registry|tool registry]], [[wiki/Dynamic Dispatch|dynamic dispatch]], and [[wiki/Capability Discovery|capability discovery]]. - **`$X` and `$R` hold scoped values, buffers, and runtime state.** Modern translation: an execution context, namespaced state store, and [[wiki/Agentic Memory|agentic memory]]. - **Set, program, document, site/account, and system scopes.** Modern translation: hierarchical context, lexical scope, and a [[wiki/Layered Context Architecture|layered context architecture]]. - **Named blocks, files, variables, panels, and remote resources are targets.** Modern translation: [[wiki/Resource Addressing|resource addressing]] and addressable document regions. - **`_do_as` projects one value into HTML, CSV, XML, RSS, or a table.** Modern translation: [[wiki/Output Projection|output projection]], serialization, and [[wiki/Content Negotiation|representation selection]]. - **Stored values can be retrieved elsewhere and transformed again.** Modern translation: a document dataflow system, scoped state, and composable transformation graph. - **Do programs call controlled X capabilities.** Modern translation: tool use through [[wiki/Permissioned Capability|permissioned capabilities]] and a [[wiki/Host Adapter|host adapter]]. - **An instruction can span retrieval, transformation, storage, and insertion.** Modern translation: an [[wiki/Agentic Workflow|agentic workflow]] and declarative orchestration. - **Files and links provide identifiers while details are loaded as needed.** Modern translation: [[wiki/Just-in-Time Context|just-in-time context]] and [[wiki/Progressive Disclosure|progressive disclosure]]. - **Markdown, variables, targets, and source relations form a navigable structure.** Modern translation: a [[wiki/Document Graph|document graph]] and [[wiki/Knowledge Graph|knowledge graph]]. - **The same Do program is intended to run through several hosts.** Modern translation: a portable workflow specification and [[wiki/Model Context Protocol|protocol-style interoperability]]. - **Execution plans, provenance, and generated regions make reruns safe.** Modern translation: [[wiki/Durable Execution|durable execution]], auditability, and [[wiki/Reconciliation Loop|reconciliation]]. The compact translation is: **XDO was a Markdown-native, document-oriented workflow DSL and human-readable IR for validated tool use, scoped state, resource addressing, data transformation, output projection, and portable execution.** Its closest current AI vocabulary is [[wiki/Context Engineering|context engineering]] plus an agent workflow runtime: the document supplies structured context, the model or person proposes intent, a constrained plan selects tools, and a permissioned host executes it. ### Relation to MCP, agent skills, and Structured Outputs The architectural comparison is strong, but it must be stated precisely. *(I did not recognize this correspondence myself until Gemini pointed it out during this reconstruction.)* XDO was **not** an early implementation of the [[wiki/Model Context Protocol|Model Context Protocol]]. MCP is a later standardized host/client/server protocol using JSON-RPC to share contextual information, expose resources, prompts, and tools, and compose AI integrations. XDO did not implement that wire protocol. What it did anticipate was the architectural need beneath it: stable capability names, discoverable operations, addressable resources, scoped context, application adapters, controlled effects, and workflows composed above the host machinery. XDO was also **not** the same mechanism as [[wiki/Structured Outputs|Structured Outputs]]. Structured Outputs constrains a language model's response to a supplied JSON Schema. Historical DoLang parsed a human-readable command language into interpreter-managed structures and normalized its own variables, commands, state, and projections. The modern connection is direct and useful: a model can translate natural language into a schema-valid Do plan; the Do parser can validate the plan's semantics, scopes, targets, and permissions; and the deterministic runtime can execute only registered capabilities. The Shazam/X practice of storing capabilities as small, separately discoverable files is likewise analogous to [[wiki/Filesystem-Based Agent Skills|filesystem-based agent skills]], particularly their present use of [[wiki/Progressive Disclosure|progressive disclosure]]: expose names and descriptions first, then load detailed instructions and resources only when required. Again, this is an architectural correspondence rather than a claim of literal product ancestry. Together these designs support a stronger philosophy of authorial agency: > **Keep the canonical document local, readable, portable, and useful without the machine. Let the machine propose or perform declared transformations through inspectable intermediate representations and permissioned capabilities. Do not make an opaque platform the sole interpreter, custodian, or operational memory of the author's work.** In this qualified sense, **maximum utilization, minimum dependency**, structured context, and [[wiki/Human-Readable Intermediate Representation|human-readable IR]] are resilience mechanisms against absorptive machine authority. The model can assist with interpretation and planning without absorbing ownership of the knowledge or replacing the deterministic contracts that make the work reproducible. Contemporary specifications used for this comparison: - [Model Context Protocol specification](https://modelcontextprotocol.io/specification/draft/index) - [OpenAI Structured Outputs](https://platform.openai.com/docs/guides/structured-outputs) - [OpenAI agent skills](https://developers.openai.com/codex/skills) The analogy has limits. XDO's historical interpreter was deterministic and convention-driven; current language models are probabilistic. The strongest continuation keeps the model above the DSL—proposing Do—rather than allowing the model to replace the contracts and runtime beneath it. See [[wiki/Modern Technical Translation Map|Modern Technical Translation Map]] for the shared vocabulary. ## The linguistic pipeline The design was deliberately layered: ```text natural-language intention ↓ constrained DoLang instructions ↓ parser and intermediate representation ↓ validation, scope, and permissions ↓ registered XDO verbs ↓ XApps, XLib, and host capabilities ↓ generated or transformed Markdown ↓ rendered document or published artifact ``` DoLang was therefore not trying to replace PHP, Perl, JavaScript, or Python. It occupied the constrained middle between ambiguous human expression and deterministic application behavior. In modern language, DoLang is an [[wiki/Intermediate Representation|intermediate representation]] for [[wiki/Natural Language to DSL Compilation|natural-language-to-DSL compilation]]. A person or model can propose a small program; the program can be inspected and validated; the host can authorize only the required capabilities; the runtime can then execute deterministic operations. The AI proposes. The contract validates. The host authorizes. The runtime executes. ## Names and historical layers The archive contains several related names and experiments: - **Do** — the short human-facing name for the instruction language; - **DoLang** — the clearest name for the language as a language; - **XDO / `xdo`** — the language in relation to the X architecture; - **`x_do.php`** — the principal PHP interpreter/dispatcher found in mature recovered implementations; - **`!do-md`** — a dedicated executable-Markdown parsing experiment; - **`_do_ext`** — separately loadable language extensions; - **`_do_as`** — output and representation adapters: do the result *as* HTML, CSV, XML, a table, RSS, or another named form. These names should not be flattened into a claim that every historical directory was one identical implementation. They represent stages and surfaces of the same design lineage: human-readable instructions, parsed into bounded semantic operations, executed through the reusable X environment, and returned to a document or publishing system. ## Executable Markdown as the language surface The surviving material shows multiple experimental syntactic forms, including constructions resembling: ```markdown [!]( ... !) [do!]( ... !) do! ... !! [do](...) ``` Two especially readable authoring forms emerged in the design discussions. Inline actions could appear within ordinary Markdown: ```text [do](load a value, transform it, place the result here) ``` Longer workflows could occupy a named section: ```text ## Do! Build the result load the source transform the records apply the template insert the output into the target done! ``` The punctuation changed while the model was being explored. The stable concept was more important than any single enclosure: readable documents should be able to declare transformations without becoming conventional application source files. The executable forms were tested inside Markdown containing headings, prose, lists, tables, links, inline code, and ordinary paragraphs. That placement was not incidental. The interpreter had to recognize protected executable regions, process them at the proper stage, preserve ordinary Markdown, and return results that could participate in subsequent rendering. ## The command vocabulary Recovered commands and tests include operations such as: ```text get add append prepend clear clearto to assign load save savevar savesetting run loop loopdata toarray csv lines kvgroups webpaths replace rand lipsum uc lc tc dblist fakedata ``` The vocabulary spans several semantic domains: | Domain | Representative operations | |---|---| | State and buffers | `get`, `to`, `assign`, `clear`, `clearto` | | Text transformation | `add`, `append`, `prepend`, `replace`, `uc`, `lc`, `tc` | | Structured input | `csv`, `lines`, `kvgroups`, `toarray` | | Iteration | `loop`, `loopdata` | | Persistence | `load`, `save`, `savevar`, `savesetting` | | System and application access | `run`, `webpaths`, `dblist` | | Generation and testing | `rand`, `lipsum`, `fakedata` | This breadth is characteristic of an orchestration DSL. The commands do not reproduce a host language. They expose the operations useful to a document author, CMS process, or application workflow. ## Documents as data containers Embedded structures such as the following allowed data to live beside the instructions consuming it: ```text lines: kvgroups: csv: ``` The same Markdown file could therefore hold: - prose explaining the operation; - source records; - named variables; - row or page templates; - executable steps; - a destination block; - generated output. ### Programs as data—demonstrated storage, experimental execution The `!do-md` test document explicitly experiments with treating a named program as data: ```text PROGRAM: lipsum(100) uc do: savevar(PROGRAM) ``` The dump shows `PROGRAM` normalized as the global variable `program`. It then shows `savevar` resolving `PROGRAM` as a global variable operand with `v_isvar=1`. The resulting file survives at `_do/program.txt` and contains: ```text lipsum(100) uc ``` This establishes that an executable instruction sequence could be named, resolved as a value, and persisted through the document runtime. It is concrete evidence for **programs as data** and for portable, reusable workflow fragments. A later test attempts: ```text do: run(PROGRAM) ``` In this particular dump, however, `run` is classified as a set-scoped variable rather than a resolved command and produces no program output. The artifact therefore supports two different claims with different evidence status: - **demonstrated:** a program body can be stored as a named value and persisted to a file; - **experimental and incomplete in this snapshot:** that stored body can subsequently be invoked through `run(PROGRAM)` as a first-class executable workflow. The distinction is important. The architectural intention is clear, but the dump lets the record remain precise about which half of the experiment was actually operational. Loop expressions could address current record fields through contextual names such as: ```text !loop.title ``` The document became a compact bundle of data, instructions, templates, and presentation without losing its ability to be read or versioned as plain text. ## A recovered working example One routed Markdown test defines: - an HTML wrapper for a data table; - an HTML row template; - a CSV dataset; - a four-command program. The program loops over the CSV records, maps each record into the row template, loads the outer table, replaces its row placeholder, and writes the result to a target variable. A larger companion document contains exactly 1,000 CSV records and runs the same four-command program. The file is 1,050 lines long. This demonstrates that the concept was not confined to a one-line string substitution; it was being exercised as a document-contained data-transformation pipeline. ```text CSV block ↓ loop through records ↓ map fields into row template ↓ insert rows into table wrapper ↓ emit named result ``` ## Interpreter architecture: program → set → command The mature recovered PHP engine is hierarchical rather than flat: ```text PROGRAM ↓ SET ↓ COMMAND ``` A document-level execution was not merely split into arbitrary lines followed by direct PHP calls. The parser created programs, ordered sets, and commands. Each layer carried buffers, variables, parameters, system metadata, and scope. The recovered schema includes: | Field | Recovered role | |---|---| | `lc` | normalized lowercase/canonical representation used in resolution | | `rc` | recovered/raw representation retained by the parser | | `isf` | whether the resolved item is a callable Do function | | `isv` | whether the item resolves as a Do variable | | `isvable` | whether a structure can participate in variable resolution | | `scp` | scope metadata; initialized to program scope in the mature engine | | `sys` | interpreter/runtime metadata associated with the structure | `DEF_SCHEMA_KV` recursively applies this model to keys and values. That is significant: recursion exists not only in callable behavior but in the representation of the language itself. XDO was moving toward a structured [[wiki/Intermediate Representation|semantic representation]], not an unstructured sequence of shell-like strings. ### `x_dump.txt` as a materialized execution record The `!do-md` directory contains a particularly valuable artifact: `x_dump.txt`, whose filesystem modification timestamp is **May 6, 2017**. The file is not another copy of the source document and not a speculative schema. `do-md.php` writes it from `print_r($X, 1)` after the Do preparation and execution passes have processed the test Markdown. It is therefore a materialized snapshot of the host and interpreter state produced by an actual run. The snapshot contains thirteen executable regions. Each region has a generated UID, an information structure, variables, one or more sets, parsed list entries, and buffers. The surrounding runtime retains shared variables and host configuration. ```text $X ├── domain ├── _do │ ├── do_order_index │ ├── do_id_index │ ├── do_data │ │ └── UID │ │ ├── info │ │ │ ├── uid │ │ │ ├── buf │ │ │ ├── vars │ │ │ └── title │ │ └── sets │ │ ├── key │ │ ├── val │ │ ├── vars │ │ ├── buf │ │ └── list │ ├── do_glo_buf │ └── do_glo_vars └── host resource paths ``` This matters because it shows both the parsed program and the effects of execution. Raw values remain visible, normalized semantic metadata sits beside them, buffers contain produced results, and `do_glo_vars` records the shared state left after the document has run. The dump is consequently evidence of a functioning intermediate representation and execution environment—not merely evidence that such a structure was planned. #### Ordered node registry The runtime maintains two complementary indexes: ```text do_order_index[0] -> 0-2nfnovxn3w do_id_index[0-2nfnovxn3w] -> 0 do_data[0-2nfnovxn3w] -> parsed region and runtime state ``` `do_order_index` preserves the sequence in which executable regions occur in the document. `do_id_index` provides the reverse lookup from a generated UID to its position, while `do_data` stores the addressable node itself. The UID combines an ordinal prefix with a random suffix, so this snapshot demonstrates **run-local unique identity**, not yet a persistent identity guaranteed to survive reparsing. In contemporary terms, this is an **ordered node registry**: identity and execution order are related but represented separately. That is a useful foundation for targeting, dependency tracking, provenance, selective re-execution, diagnostics, and eventually stable generated regions. A modern XDO should preserve this separation while replacing transient random IDs with explicit persistent source identities where reconciliation across runs is required. #### Genuinely semantic resolution Each parsed list entry retains more than textual tokens. The dump records fields such as: | Field | Evidence in the dump | |---|---| | `key`, `val` | source spelling and source value | | `k_lc`, `v_lc` | normalized key and operand names | | `k_type` | whether the key resolved as a command or variable | | `k_scope`, `v_scope` | resolved scope of a declaration or operand | | `v_isvar` | whether the operand resolved as a variable reference | | `v_type` | additional value or command classification | The differences are visible in actual evaluated instructions: ```text savevar(PROGRAM) k_type -> cmd v_lc -> program v_scope -> glo v_isvar -> 1 rand(25) k_type -> cmd v_lc -> 25 v_scope -> empty v_isvar -> empty get(LIPTEST) k_type -> cmd v_lc -> liptest v_scope -> glo v_isvar -> 1 ``` The parser is distinguishing a literal operand from a reference, assigning semantic scope, classifying declarations and operations, and preserving a normalized canonical name. That is a true resolution layer between syntax recognition and execution. It is more accurately described as an early semantic IR than as macro substitution. The root of the same snapshot contains `domain=pub.dev` and separate `path_sys`, `path_app`, `path_mod`, `path_inc`, `path_res`, `path_acs`, and `path_glb` values. Those host paths are old and environment-specific, but structurally they show that the Do runtime lived inside a supplied host context with differentiated application and resource roots. That supports the modern host-adapter interpretation while also identifying hard-coded environment paths as a portability problem to remove. ## Multi-pass processing The mature interpreter contains an opening pass and explicit multi-pass stages for programs, sets, and commands. The earlier `!do-md` experiment likewise contains separate preparation passes for Markdown lists and inline forms. The intended order can be described as: ```text Markdown source ↓ recognize protected Do regions ↓ parse programs, sets, commands, keys, and values ↓ resolve variables, functions, parameters, and scope ↓ execute registered transformations ↓ apply output representation and named targets ↓ return controlled results to Markdown ↓ render ordinary Markdown ``` This [[wiki/Multi-Pass Processing|multi-pass processing]] prevents every concern from being mixed into one regular-expression substitution. Recognition, normalization, resolution, execution, and rendering are distinct stages even when the historical code still contains unfinished branches. ## Runtime state: `$X`, `$R`, variables, and buffers The interpreter maintains managed runtime state rather than treating PHP's own variable scope as the language model. The recovered implementation places Do state under `$X['_do']` and constructs a working runtime structure commonly referenced as `$R`. That structure contains configuration, programs, sets, commands, variables, buffers, and system metadata. Conceptually: ```text CMS / host runtime │ │ shared X environment ▼ $X │ Do runtime state │ ├── global variables and buffers ├── program variables and buffers ├── set variables and buffers ├── command parameters and metadata └── loop or recursive evaluation context ``` This is closer to a hierarchical environment model than a collection of global PHP variables. A command evaluates inside an interpreter-managed context. Nested interpretation receives a context. Loops introduce contextual bindings. Application and library calls cross outward through the shared X environment only when the language requests them. The result is a useful separation between transient computation and persistent or shared application state. ## Scope and addressability Scope is a DoLang property, not incidental leakage from PHP. The mature recovered interpreter establishes three concrete internal scope labels: - `set` — a set-local value; - `prg` — a program-level value; - `glo` — a shared/global runtime value. The recovered scope resolver also gives us a historical case convention: | Identifier form | Scope in this recovered implementation | |---|---| | `_name` | `set` | | `name` | `prg` | | `Name` | `glo` | | `NAME` | `glo` | This corrects the earlier reconstruction: the case-to-scope direction is not wholly unknowable. It is visible in the mature recovered snapshot. However, it should be recorded as **historical implementation behavior**, not yet frozen as the permanent law of a modern DoLang. The same source contains a comment showing that lowercase values may earlier have been considered for set scope, and not every mixed-case shape is explicitly resolved. Exact shadowing and write-through behavior in all nested contexts also requires further fixture-based recovery. The broader modern scope model remains useful: 1. **run or block scope** — temporary values for one execution; 2. **document scope** — values and named blocks shared within one Markdown document; 3. **account scope** — persistent values shared by authorized documents in one account; 4. **system scope** — explicitly granted values or templates available across accounts or domains. These modern names should map onto explicit capabilities and storage contracts rather than depend on capitalization alone. Case can remain concise authoring sugar while the parsed representation stores scope directly. ### Worked proof: casing, semantic scope, and cross-region dataflow The dump contains a controlled scope experiment using three spellings of the same canonical name: ```text aa(aa content) Aa(Aa content) AA(AA content) ``` The runtime resolves them as follows: | Source identifier | Canonical name | Resolved scope | |---|---|---| | `aa` | `aa` | `set` | | `Aa` | `aa` | `prg` | | `AA` | `aa` | `glo` | This earlier `!do-md` convention differs from the mature interpreter's `_name`/`name`/`Name` convention documented immediately above. That is evidence of the casing law evolving between implementation branches, not evidence that casing was irrelevant. The durable design is visible in both versions: source form carries scope semantics while a separate canonical lowercase name supports symbol resolution. A modern grammar should preserve the semantic distinction explicitly rather than accidentally freezing either historical shorthand. Normalization did not erase the author's scope instruction: the canonical name enabled resolution while the original form determined where the value belonged. Later commands demonstrate the resolver following those bindings. `add(Prgvar)` becomes a program-scope variable reference; `add(GLOBALVAR)` becomes a global-scope variable reference; and the resulting program buffer contains the transformed program value followed by the global value. The strongest result in `x_dump.txt` is a working cross-region dataflow experiment. One executable region performs: ```text lipsum(50) uc clearto(LIPTEST) ``` After execution, the region's local buffer is empty and `do_glo_vars['liptest']` contains the generated uppercase text. A later executable region—separate in document order—performs: ```text get(LIPTEST) ``` The later command resolves `LIPTEST` to canonical `liptest`, marks it `v_scope=glo` and `v_isvar=1`, and places the earlier region's uppercase value into both the set buffer and the program information buffer. The same pattern is exercised with `RANDTEST`; compact inline and multiline forms both retrieve the shared value successfully. This proves that executable regions were not isolated substitutions. They could communicate through interpreter-managed document state across region boundaries. It also demonstrates buffers at semantic levels: ```text command transforms current value ↓ set buffer ↓ program information buffer ↓ clearto(NAME) moves the result into shared state ↓ later region resolves NAME and retrieves the result ``` In modern terms, the document already possessed an ordered execution context, namespaced shared memory, canonical symbol resolution, and a small dataflow graph. The later vision of document, account, and system scopes was therefore an extension of behavior already working locally—not an idea invented only after the fact. ## Named targets The long-term addressability model extended beyond variables. A transformation could target regions such as: ```text header sidebar footer panel-a a named Markdown block a named block in another document a persistent account value an authorized system template ``` This is the role of a [[wiki/Named Target|named target]]: the program says *where* a result belongs without hard-coding the rendering host's DOM, PHP template, or filesystem layout. A local transformation could push a value to another block in the same document. A document-scoped value could be retrieved elsewhere in that document. Account-scoped values could be reused by authorized documents in the same account. System-scoped templates could be made available across domains when permission allowed it. That architecture made templates portable. The document described semantic destinations; the host adapter resolved those destinations in a website, editor, static generator, local application, or Obsidian plugin. ## Loops and iteration frames Iteration is where DoLang's scope model becomes especially visible. `loop`, `loopdata`, and related experiments were not merely textual repetition facilities. Loop evaluation exposes iteration-specific data through the interpreter environment. ```text PROGRAM └── SET └── LOOP ├── iteration 1 context │ └── commands and templates ├── iteration 2 context │ └── commands and templates └── iteration n context └── commands and templates ``` Expressions such as `!loop.title` resolve against the current record. The current row is therefore an interpreter binding, not a raw PHP `foreach` variable exposed directly to the document. The mature `x_do_loop` implementation supports concepts including: - the value being iterated; - `with` and `as` transformation modes; - `map` field mappings; - `to`, `totop`, and `tobottom` destinations; - template-backed transformation; - insertion into buffers or named values. What remains to be recovered through focused tests is the exact mutation law for every nested loop: when an assignment shadows a parent binding, when it writes through, and how explicit promotion should work. The architecture is established; every edge case is not. ## Dynamic dispatch: `x_do_*` as the verb registry One of the most elegant implementation features is convention-based command dispatch. The interpreter normalizes the command name and constructs a handler name: ```php $cmd_func = "x_do_{$command}"; ``` It then checks whether the function exists and invokes the resolved handler. The older `!do-md` code preserves the same idea through logic equivalent to: ```php strtolower("x_do_$do_key_name") ``` Thus: ```text loop → x_do_loop() save → x_do_save() get → x_do_get() append → x_do_append() ``` The dispatcher is a [[wiki/Dynamic Dispatch|dynamic dispatch]] membrane between the small language and the much larger host environment. New semantic verbs can be added through `_do_ext` without continually expanding the grammar. Historically, the registry was implicit: a function became callable when a matching `x_do_*` name existed. A modern implementation should make that an explicit [[wiki/Capability Registry|capability registry]] with input rules, normalized output, permission requirements, purity/effect metadata, and conformance fixtures. ## XApps and XLib XApps and XLib are the point where the executable document becomes an orchestration surface over the larger system. - **DoLang** defines what an executable document is permitted to say. - **XApps** exposes application-level operations or modules. - **XLib** provides reusable lower-level capabilities. - **The host adapter** controls access to files, network requests, CMS objects, editor regions, or other environment-specific effects. ```text ┌────────────────┐ │ XApps │ │ application │ │ capabilities │ └───────▲────────┘ │ Markdown → DoLang → x_do dispatcher │ ┌───────▼────────┐ │ XLib │ │ reusable host │ │ capabilities │ └────────────────┘ ``` The surviving site runtimes load the XApps loader, while the mature XDO configuration locates `_do_ext`, `_do_as`, and templates through the X library paths. This gives artifact support to the architectural relationship even where every experimental bridge file has not yet been recovered. The important separation is conceptual: a document does not need arbitrary PHP access merely because it needs an application operation. That operation should be exposed as a bounded semantic verb or [[wiki/Permissioned Capability|permissioned capability]]. ## Buffers and transformation pipelines Execution is organized around intermediate state and document fragments. A command can load data into a buffer, transform it, loop over records, substitute contextual fields, invoke another operation, and send the result to another variable, buffer, or named target. ```text data ↓ Do command ↓ buffer ↓ loop, transform, or template ↓ substitution ↓ output buffer ↓ Markdown or another projection ``` The natural unit of computation is often not a number or class instance. It is a **document fragment** or structured dataset moving through an inspectable pipeline. Generated output may itself require another resolution pass. The program/set/command hierarchy, recursive key-value schema, and multiple execution stages were mechanisms for keeping that recursive interpretation structured rather than reducing it to uncontrolled string replacement. ## `_do_as`: output projection as a first-class layer `_do_as` means more than “apply a template.” It expresses the requested [[wiki/Output Projection|output projection]] or **representation of a result**: ```text do this data as HTML do this data as CSV do this data as XML do this data as a table do this data as RSS ``` This produces a cleaner architecture: ```text source data ↓ Do transformation ↓ normalized intermediate value ↓ AS adapter ├── HTML ├── CSV ├── XML ├── table / datatable ├── RSS └── another registered representation ↓ named target or output buffer ``` The archive contains several concrete forms of this idea: - `_do_as/rss.as`; - `_do_as/rss.dtp`; - callable adapters following `x_do_as_tpl_*`; - a `datatable.tpl.php` projection with responsive table generation; - site-theme adapters that can override system adapters; - field maps that connect dataset fields to projection inputs. The mature engine searches for a site-theme adapter first, then a system `_do_as` adapter, then an already loaded function. This is a meaningful precedence rule: shared defaults can be overridden by the current site's presentation layer without altering the semantic program. That makes `_do_as` an early form of content negotiation and projection dispatch. The Do program describes the transformation; the `as` adapter decides how the normalized result should be represented. Representation was not necessarily the end of the pipeline. A projected value could be displayed immediately, sent to a named target, or stored for later retrieval. A stored value could subsequently be loaded, transformed again, and projected into a different representation. The complete lifecycle was therefore: ```text acquire or declare data ↓ transform ↓ store in a chosen scope and/or representation ↓ retrieve from another block, document, site, or authorized global context ↓ transform again ↓ project as HTML, CSV, XML, table, RSS, or another form ↓ display or send to one or more named targets ``` This separates three concerns that are often collapsed in conventional CMS code: 1. **semantic value** — what the data means; 2. **storage scope** — where the value can be retrieved: document, site/account, or global/system; 3. **output projection** — how the value should be represented at a particular destination. Because those axes are independent, the same stored value can be reused many times without repeating its acquisition or original transformation. One document can produce it, another authorized document can retrieve it, and each consumer can request the representation appropriate to its target. ## Persistence and store topology The archive preserves file-backed values, templates, and three important store roots: ```text sys db site ``` These names correspond to different addressability and reuse boundaries. The surviving LDEV store contains both a scalar value and a reusable template, while the surrounding runtime configures the store paths before running opening and multi-pass processing. The historical goal was broader than a single request buffer. A value could be temporary, stored in a document variable, stored as a site or account variable, or made available across sites through an explicitly shared global/system layer. It could then be retrieved from any authorized context, transformed again, displayed, or sent to a named target. Storage did not have to erase representation. A value could remain structured for later transformation or be persisted in a selected form—such as Markdown, HTML, CSV, XML, RSS, lines, or key-value groups—when that representation itself was the reusable artifact. For a modern implementation, persistent writes should be represented as declared effects: ```text read document value write account value read system template emit site target ``` Each effect should carry a resolved target, permission, provenance record, and dry-run representation. ## Markdown as a portable application surface The document remains the authoring surface. The runtime is replaceable. A web server, local editor, static generator, desktop application, or Obsidian plugin could interpret the same validated workflow if each host exposed the same named capabilities and normalized contracts. This is where XDO meets [[projects/X and XApps - Drop the Folder and It Runs|X and XApps]]. - Do describes the workflow. - X supplies portable functions. - `_do_as` supplies output representations. - The host adapter supplies controlled environment access. - Shared fixtures verify that every runtime returns the same normalized result. The same document could therefore run in PHP on a server, in JavaScript inside an editor, through Node in a local tool, or eventually in Python or Perl—without redefining the meaning of its operations for every host. ## Natural language and LangChain experiments The eventual authoring surface could be a sentence such as: > Take this CSV data, put it in a table, store the result globally, and insert it into the named target as HTML. A deterministic translator or language model could compile that sentence into a proposed Do program. Bryant also recalls experimenting with [[wiki/LangChain|LangChain]] for this natural-language parsing layer; those notes remain to be added to the archaeological record when recovered. The safe relationship is: ```text natural language ↓ proposed Do program ↓ parsed semantic plan ↓ validation and permission review ↓ registered capabilities ↓ controlled host effects ``` Natural language is expressive but ambiguous. DoLang is constrained and inspectable. The host language is powerful but too broad to expose directly. DoLang occupies the useful middle. ## Relationship to the older preparser and Parrot concept Bryant's reconstruction identifies an older architecture in which `x_do.php` acted as a first-tier preparser: natural language or higher-level input could be reduced to Do instructions, and the resulting representation could then be passed onward to **Parrot**. That claim should be retained as design genealogy while remaining clearly separated from artifact-verified details until the corresponding Parrot source is located. The conceptual continuity is strong: ```text human expression ↓ constrained intermediate language ↓ execution substrate ``` The later executable-Markdown work brought that intermediate language directly into the CMS and document environment. The backend could change; the architectural principle remained: use a small semantic language to translate document intent into composable machine operations. ## What the DSL is—and is not DoLang's intended domain is bounded: ```text document state structured text key-value data CSV, lines, and groups templates and buffers loops and mappings persistence CMS resources application and library calls output projections content generation ``` It does not need to reproduce classes, pointers, operating-system primitives, unrestricted SQL, shell execution, or every general-purpose control structure. Those facilities exist beneath the language and should be exposed only through narrow registered capabilities. The farther down the following stack one travels, the more general and dangerous the capability becomes: ```text human / author ↓ Markdown ↓ DoLang ↓ semantic verbs ↓ XApps / XLib / host adapter ↓ PHP / JavaScript / Python / Perl ↓ storage / network / operating system ``` DoLang deliberately sits high in the stack—close to intention and far from unrestricted machinery. ## Why the historical engine should not simply be exposed The recovered implementation contains dynamic dispatch, broad file operations, database experiments, and unfinished scope branches. Those are valuable design evidence, but they are not a sufficient modern security boundary. A production continuation should not permit: - unrestricted `eval`; - arbitrary shell execution; - unconstrained filesystem paths; - arbitrary SQL; - arbitrary network destinations; - calling any host function merely because its name exists; - implicit cross-account reads or writes; - unlimited recursion or resource consumption. The historical engine is a language-design donor and executable specification source, not production code to expose directly. ## Modern continuation A modern XDO should preserve the authoring philosophy while formalizing the runtime. ### 1. Versioned grammar and semantic representation Parse the document into a versioned abstract syntax tree or other explicit [[wiki/Intermediate Representation|intermediate representation]]. Preserve raw source spans alongside normalized command names and values. ### 2. Explicit capability registry Replace open-ended `function_exists()` dispatch with a [[wiki/Capability Registry|registry]] containing: - [[wiki/Canonical Capability Name|canonical capability name]]; - aliases and deprecations; - [[wiki/Input Rules|input rules]]; - [[wiki/Normalized Output|normalized output]]; - [[wiki/Error Behavior|error behavior]]; - purity or effect classification; - required permissions; - supported runtimes; - [[wiki/Shared Fixtures|shared fixtures]]. Together these form a [[wiki/Portable Contract|portable contract]]. ### 3. Explicit scope metadata Case may remain compact authoring sugar, but the parsed program should record scope directly. Modern scopes should include run/block, document, account, and system, with explicit import, export, promotion, and shadowing rules. ### 4. Typed values and output projections Distinguish strings, record sets, key-value groups, CSV, Markdown fragments, HTML fragments, and other data types. `_do_as` adapters should accept normalized inputs and return declared output types. ### 5. Host effects as permissions Filesystem, network, persistent storage, UI targets, and application calls should be exposed as narrow [[wiki/Permissioned Capability|permissioned capabilities]] through a [[wiki/Host Adapter|host adapter]]. ### 6. Execution plans and dry runs Before execution, the runtime should be able to show: - commands to be run; - data to be read; - targets to be modified; - external effects requested; - templates and adapters selected; - expected output type; - permissions required. ### 7. Provenance and generated regions Generated output should record its source document, Do program, capability versions, inputs, and generation time. Hand-edited material should be kept outside owned generated regions so regeneration can remain lossless. ### 8. Resource limits Set limits for recursion, iteration, input size, output size, time, and network access. ### 9. Cross-runtime conformance JavaScript, Node, PHP, Python, and Perl implementations should consume the same [[wiki/Shared Fixtures|shared fixtures]] and pass the same [[wiki/Conformance Testing|conformance tests]]. This is how one Do document can preserve meaning across servers, local applications, editors, and plugins. ## Application to the Obsidian writing system The current Obsidian work makes the original XDO idea immediately practical. A document-native workflow could safely declare how to: - discover marked quotes and passages; - generate or update derived quote and passage pages; - preserve curated commentary beneath generated regions; - join source metadata; - generate posters; - project content as HTML, RSS, JSON, CSV, XML, or tables; - update indexes and search surfaces; - place results into named blocks; - create distribution feeds; - run the same transformation locally or on a server. The transformation would travel with the knowledge without surrendering the knowledge to one application. This is also the architectural answer to avoiding one-off “vibe coded” systems. A model should generate a validated Do program against known capabilities, not invent a new pile of glue code for every request. ## Why this matters XDO anticipated a problem that has become much larger: people want to tell software what outcome they want without losing the ability to inspect, reproduce, move, and audit the process. A conversational instruction can disappear into a model session. A conventional program may be too technical for the document's author. A Markdown-native workflow offers a middle layer: readable enough to live with the subject, structured enough to validate, and portable enough to execute in more than one host. The deeper XDO principle is: > **The document should be able to explain what it wants done, and that explanation should remain useful wherever the document goes.** ## What is established and what remains open ### Established by surviving artifacts - executable instructions were embedded and tested inside ordinary Markdown; - the May 6, 2017 `x_dump.txt` is a materialized post-execution snapshot containing thirteen parsed executable regions and their resulting state; - executable regions were registered through complementary order-to-UID and UID-to-order indexes, separating run-local node identity from execution order; - the mature parser modeled programs, sets, commands, keys, values, variables, buffers, and system metadata; - the `!do-md` semantic record distinguished commands from variables, normalized canonical names, resolved operand scope, and recorded whether an operand was a variable reference; - the runtime used opening and multi-pass processing; - commands dynamically dispatched to `x_do_*` handlers; - `set`, `prg`, and `glo` scopes existed in the mature implementation; - the recovered implementation mapped `_name` to set scope, lowercase `name` to program scope, and `Name`/`NAME` to global scope; - the earlier controlled `aa`/`Aa`/`AA` test independently demonstrates set/program/global resolution of three source forms sharing one canonical name and documents an earlier casing convention in the design's evolution; - separate executable regions exchanged computed values through shared `LIPTEST` and `RANDTEST` bindings, proving working cross-region document dataflow; - the runtime retained buffers and variables at set, program, and shared/global semantic levels; - `PROGRAM` was persisted as an instruction-bearing value in `_do/program.txt`; the attempted `run(PROGRAM)` invocation remained incomplete in this snapshot; - loops used interpreter-managed data, maps, templates, and destinations; - CSV-to-HTML transformation was exercised with a 1,000-record fixture; - persistent file-backed values and templates were connected to `sys`, `db`, and `site` store roots; - `_do_as` resolved site-level and system-level output adapters, including datatable and RSS experiments; - XDO paths were configured through the reusable X library environment. ### Established by Bryant's account with partial artifact support - named layout blocks and document/account/system addressability were part of the broader design; - XApps and XLib were intended as application and library boundaries beneath DoLang; - the same function abstractions were intended to align across server, local, plugin, and editor hosts; - the ultimate objective was living, state-bearing documents; - Markdown was chosen as the human-machine common surface rather than merely as a convenient storage format. ### Owner reconstruction awaiting additional artifacts - the precise genealogy connecting the earlier preparser to Parrot; - the full natural-language parser and LangChain experiments; - every historical spelling and enclosure rule; - every nested-loop shadowing and write-through edge case; - the complete historical catalog of `_do_as` projections; - the final intended casing law across every implementation branch. ### Present architectural proposal - explicit AST or semantic IR; - allowlisted capability registry; - typed values and adapters; - explicit permissions and effect declarations; - dry-run execution plans; - provenance-owned generated regions; - portable contracts and shared fixtures across languages. ## Archaeological sources Primary recovered implementation areas used for this record include: - `/Volumes/Extreme SSD/dev-pubtools/_pub_tools/scraps/_x/x_do/x_do.php` - `/Volumes/Extreme SSD/dev-pubtools/_pub_tools/scraps/_x/x_do/_do_as/` - `/Volumes/Extreme SSD/development/XX__/former/!do-md/` - `/Volumes/Extreme SSD/development/XX__/former/!do-md/x_dump.txt` — materialized host, parser, scope, buffer, and post-execution state; filesystem modification timestamp May 6, 2017 - `/Volumes/Extreme SSD/development/XX__/former/!do-md/test_data.txt` — source Markdown for the scope, persistence, inline, and cross-region tests represented in the dump - `/Volumes/Extreme SSD/development/XX__/former/!do-md/_do/program.txt` — persisted instruction-bearing `PROGRAM` value - `/Volumes/Extreme SSD/development/XX__/former/site-markbox/xapps/libs/x/_do_as/` - `/Volumes/Extreme SSD/development/XX__/former/site-markbox/sharedlib/xapps/libs/x/_do_as/` - `/Volumes/Extreme SSD/development/XX__/former/ldev/public/acs/master.ldev/_do/store/` - `/Volumes/Extreme SSD/development/XX__/former/vdev/routes/datatables/` The owner-supplied August 17 architecture reconstruction is preserved separately as: - `projects/sources/xdo-dolang-architecture-reconstruction-2026-08-17.txt` ## Related projects and concepts - [[projects/X and XApps - Drop the Folder and It Runs|X and XApps: Drop the Folder and It Runs]] - [[projects/AutoSites - Folder-Native Publishing and Distribution|AutoSites: Folder-Native Publishing and Distribution]] - [[projects/Shazam - A Portable Perl Capability Library|Shazam: A Portable Perl Capability Library]] - [[projects/Ten Years Building a Symbolic Language Engine|Ten Years Building a Symbolic Language Engine]] - [[wiki/Executable Markdown|Executable Markdown]] - [[wiki/Domain-Specific Language|Domain-Specific Language]] - [[wiki/Intermediate Representation|Intermediate Representation]] - [[wiki/Natural Language to DSL Compilation|Natural Language to DSL Compilation]] - [[wiki/Dynamic Dispatch|Dynamic Dispatch]] - [[wiki/Capability Registry|Capability Registry]] - [[wiki/Named Target|Named Target]] - [[wiki/Portable Contract|Portable Contract]]