WM-04: build an observation model that can disagree with the machine

Authored design cascade, 2026-09-15. Target: exactly WM-04. This proposes the complete build path from observation meaning to an admitted, consumed A artifact. It keeps unresolved semantic decisions visible. It is not an implemented pipeline, a measured model or permission to resume build dispatch. WM-05's cascade is its immediate consumer-side companion.

What we are trying to build

WM-05 needs to predict observations from possible states of the actual work. WM-04 must supply the observation model: what observations are supported in each state, grounded in production evidence rather than supplied as a convenient table. The accepted checklist requires independently authorized and blinded observation, exact-subject review, predeclared sampling, immutable cutoff evidence, defensible state/outcome pairs, honest coverage and refusal of unsupported assembly.

Consider the recurring handoff example. A mission is marked complete, but the next reviewer cannot tell whether its success criterion was satisfied. A registry label, the machine's confident belief and an author's success claim are three available records. None automatically supplies an independently observed state. A reviewer may correctly accept the finding that evidence is insufficient. That acceptance still cannot populate a likelihood cell.

The inherited seven-state/twelve-disposition contract constrains the existing interface. It does not justify the meanings of its labels. If those meanings cannot support independently acquired pairs, this design stops for a documented model decision. We do not silently replace the carrier, label states with belief argmax, or introduce a latent-state estimator as a workaround.

The three original seeds are Observation Can Disagree, Unobserved Is Not Zero, and Measurement Window Hygiene. Three new draft patterns supply missing arguments about prospective selection, exact review and paired-label authority. Other existing seeds connect them. This is an authoring cascade: its relations explain why the design's parts constrain each other. No numerical G or runtime firing semantics is claimed for it.

Pattern applications and their contributions

O1 — Give state and observation their separate meanings

Pattern: wm-prediction/meaning-before-carrier.

Declare the target kind, state support, observation support, reference time, grain and conditioning assumptions. For each state, name the criterion that could justify assigning it, and evidence that could contradict it. Explain whether the observation is contemporaneous with the state or an eventual disposition; an eventual disposition cannot be paired with an earlier state without a declared temporal interpretation. Output: a versioned semantic declaration, with unresolved entries explicit.

O1 constrains both the observer's rubric and the WM-05 consumer's expected domain. Review must challenge whether a proposed A is actually describing observations given state, or has smuggled in transition dynamics, preferences or an action-specific success model. Any extra conditioning requires explicit model authority and compatible consumers.

O2 — Preserve the possibility of disagreement

Pattern: wm-prediction/observation-can-disagree.

Use O1's criteria to design the observer's task. Its admissible inputs must allow a result the current belief would disfavor. The state annotator must not see the posterior, the model's predicted answer or the close disposition. A separate acquisition role can retain those records for later joining without exposing them to the annotator. Redacted material retains a link to its frozen source; it is not silently rewritten evidence.

Output: an observer input contract and access boundaries. Calling an agent independent does not establish independence: retain who authorized it, what it received, and its relationship to the work's author. Distinct review is separately required at O6.

O3 — Choose the cohort before seeing results

New pattern: wm-prediction/choose-observations-before-outcomes.

Freeze the eligible production population, inclusion and sampling procedure, window and stopping rule before outcome reveal. Each selected occurrence receives an acquisition entry even if its evidence later proves absent. Output: the prospective acquisition plan and selection ledger. No concrete sample size or success quota is invented here; adequacy and stopping criteria need reviewed authority before collection.

Selecting only successful closes after looking at them cannot be repaired by an excellent observer. A changed plan starts an explicitly identified successor cohort; it cannot retroactively redefine the old denominator.

O4 — Admit evidence from the right window

Pattern: aif/measurement-window-hygiene.

Apply temporal discipline to O3's selected occurrences: distinguish the subject's reference time, evidence eligibility cutoff, acquisition time and annotation time. Freeze the admitted literal evidence by identity and content hash. Later annotation may inspect already admitted pre-cutoff evidence; later facts cannot be imported as if they were available at the cutoff.

For measurements whose validity depends on settling or before/after comparison, declare the appropriate timing and confound rules. The seed's two-read/epsilon example is not a universal WM-04 sampling rule. Confounded or transient records may be ineligible for a likelihood count, but remain in the acquisition ledger with reasons. Dropping them from all records would hide selection and missingness.

Output: a frozen manifest plus readable admitted bytes, or a typed acquisition failure. A manifest alone does not prove the reader acquired those bytes or that they support a criterion.

O5 — Classify without forcing an answer

Pattern: aif/structured-observation-vector.

The observation record binds entity, occurrence, reference time, rubric/model revisions, observer authority, input-manifest identity and the finding. Permit categorical classification only where the rubric supports it. Retain insufficient, ambiguous and conflicting findings as distinct outcomes of the observation workflow, not as extra states or measured likelihood cells.

Output: an immutable observation proposal with explicit criterion-to-evidence references. The existing AIF pattern supplies structured-observation discipline; its particular channels or numerical conventions do not override WM-04's contract.

O6 — Review these exact bytes

New pattern: wm-prediction/review-the-exact-observation.

A distinct authorized reviewer checks the exact observation, admitted evidence, rubric application and observer authority. The review states what it accepts: a categorical label, a finding of insufficiency, or neither. Output: a decision bound to the complete observation's hash and exact subject. Changed evidence, findings or rubric requires a successor record and fresh review. Keep rejected and superseded records visible, with only the admitted revision eligible downstream.

O6 is not a ceremonial second signature. Negative controls must show that a reviewer acceptance for another occurrence, another cutoff or a previous version cannot authorize the current label.

O7 — Join independently justified state and outcome

New pattern: wm-prediction/pair-state-and-outcome-without-belief-labels.

Join the accepted state classification with the admitted observation/disposition at the exact entity, occurrence, reference-time relation and compatible model domain specified by O1. Both sides need authority. A retained disposition is not automatically a measured outcome of the required kind; its production origin and interpretation must be checked. State annotator blinding is preserved until classification and review have been frozen.

Output: either an eligible pair referencing both records and their acceptance, or an explicit refusal naming the missing authority. Deduplicate by the declared observation unit, not by filename. Repeated events and correlated observations require an explicit sampling interpretation; they cannot silently inflate a denominator. If state labels cannot be independently established, return to O1. An estimator cannot cure that problem by counting posterior argmax labels.

O8 — Account for the entire selected population

Pattern: wm-qualifying-run/complete-acquisition-universe.

Reconcile every O3 selection against O4–O7. Output: a census of selected, acquired, reviewed, pair-eligible, excluded, unavailable and unresolved occurrences with reasons and source references. These categories form an explicit accounting scheme; where categories overlap, the schema must say so rather than present their sum as a total.

A list of accepted pairs cannot establish what acquisition missed. Compare the ledger with an independently retained selection record and detect omitted entries. Authentic missing history stays missing. This applies the acquisition pattern to WM-04's cohort, not to completing the entire run certificate.

O9 — Expose sparse data before estimating

Pattern: wm-prediction/unobserved-is-not-zero.

From O7's eligible pairs and O8's census, retain counts per state/outcome, row denominators, uncovered states and rejected-pair reasons. Output: a coverage report and explicit collection needs. No eligible observations for a row is different from zero occurrences of one outcome in a sampled row.

Additional collection must have its own prospective authority; the first plan cannot be extended opportunistically until a desired result appears. Missingness may depend on outcome or state. Record that limitation and refuse a claim of representative coverage until the relevant assumptions have been reviewed. Merely filling every row does not establish statistical adequacy.

O10 — Construct only the model the evidence licenses

Pattern: wm-prediction/probability-laws-with-runtime-meaning.

For the contract's categorical frequency estimator, calculate each sampled row from eligible counts and its denominator. Preserve exact count ratios in the measurement artifact and explicitly admitted numeric conversion at the runtime boundary. A zero-denominator row produces a typed absent-row result, not a uniform distribution. An observed zero remains zero. Joe's recorded measured-A ruling says no smoothing or invented mass.

For an admitted categorical model, the proposed estimator is A(o | s) = n(s,o) / N(s), where N(s) counts the eligible observations assigned to state s and equals the sum of that row's outcome counts. This applies only where N(s) > 0 and the paired-label, sampling and domain assumptions above hold. Preserve the original counts so a reviewer can reproduce every entry.

Output: either an admitted A artifact under :observed-estimate, with measurement records, supports, estimation-rule revision and review, or a refusal with missing obligations. Any future proposal for priors or another estimator requires explicit authority; this cascade does not grant it. Normalization proves a probability law, not empirical adequacy. A finite number of observations is not by itself proof that the model generalizes.

O11 — Preserve that model into prediction

Pattern: wm-prediction/outcome-meaning-across-models.

Pass the exact admitted A, measurement provenance, support ordering and model revision to assembly and WM-05's observation predictor. Reject a stale revision, mismatched domain, partial table or consumer fallback. Output: a retained construction/consumer receipt identifying the actual artifact read. An unused model file is insufficient.

This is the WM-04 interface acceptance boundary. Full policy prediction and scorer integration remain WM-05 obligations; preferences remain their own obligations. WM-04 must nevertheless show that its artifact is usable under that exact contract, and later establish its real production consumption.

Composition: why this is a cascade

O1 constrains O2 and O7; O2 and O3 jointly constrain O4 and O5. O4 supplies the frozen evidence to O5. O5 supplies an exact proposal to O6. O6 and the separately admitted outcome supply O7. O3 and O4–O7 supply O8's completeness check. O7 and O8 jointly supply O9: neither valid pairs alone nor a complete list of missing records supplies an estimator. O9 permits or blocks O10; O10 supplies O11.

Feedback is equally substantive. O7 can expose an unclassifiable state and return the problem to O1. O9 can expose a biased acquisition procedure and require a new O3 cohort. O10 can reject the proposed support or arithmetic. O11 can reveal that the model describes a different observation from the consumer's. These are review relations, not automatic retries or permission to change the specification.

Proposed build interfaces

These are proposed artifact names and component responsibilities, not claims that corresponding modules already exist. Reuse compatible existing validators and retention code after checking their actual contracts.

All records need schema version, immutable identity, authority references and content pins. Mutable paths alone are insufficient. Corrections append successor records with explicit supersession; replay selects the admitted revision without deleting the audit history. Reader/writer integration must preserve atomic publication of a complete artifact: consumers cannot observe half an update and silently use a mixture of revisions.

One occurrence through the design

Take a preselected production handoff occurrence, not a hand-picked success. O1 declares what state at the relevant time the rubric is testing. O4 freezes the allowed criterion evidence. The state annotator receives that material without posterior or close disposition. It finds the criterion evidence insufficient; a distinct reviewer accepts that exact finding.

O7 therefore produces no eligible pair even if a close disposition exists. O8 still accounts for the selected occurrence. O9 records the unclassified observation and unresolved coverage. O10 refuses unsupported model assembly. This is a correctly functioning refusal path, not evidence that A is built.

A future occurrence can contribute only if both categorical state and observation receive the required independent authority and exact join. Even then, one accepted pair only changes the licensed counts; it does not complete seven-state coverage or establish adequacy. This design deliberately supplies no pretend positive observation.

Build order and acceptance evidence

1. Review semantics and acquisition together (O1–O4). Retain a worked rubric application that permits disagreement, the blinding boundary, the prospective cohort and rejection examples. Unresolved carrier meaning, reference time, sampling/stopping or adequacy authority blocks dependent implementation decisions.
2. Build the evidence-to-pair path (O4–O7). Tests reject posterior leakage, unauthorized observer origin, missing admitted bytes, wrong occurrence/cutoff, modified subject, self-review, and insufficiency promoted to a label. A reviewed observation must replay from retained inputs with the same eligibility result.
3. Build coverage and estimation (O8–O10). Test missing selected members, duplicate pairs, obsolete acceptances, unseen rows versus observed zeros, incompatible supports and conversion admission. Recompute counts from the retained pair identities. Constructed fixtures validate mechanics only.
4. Join the real consumer (O11). A mismatch test must fail at the actual model-assembly boundary. A later authorized production receipt must identify the very A artifact used. Logging A beside a different model does not pass.
5. Acquire and review real observations under the approved plan. Retain the first complete prospective observation, then the coverage and adequacy evidence needed for the admitted model. If data remain sparse, retain and authorize the needed follow-on collection. Do not turn passing tests into measured-A completion.

Independent conceptual review must settle the open semantics; independent code review must assess implementation; independent observation review must assess each submitted finding. These roles serve different purposes and one acceptance does not replace another. This document assigns responsibilities, not agents or live jobs.

Sources and current limits

• Checklist WM-04: governing obligation and bounded historical evidence, including zero qualifying statuses in the cited exercise and zero licensable pairs in the cited historical census. Those counts were not rerun for this design.
• futon2/holes/labs/wm-contract/WORK-REMAINING.md, row 14, “A-CONTENT RULED BY JOE”: production :observed-estimate, real data with certificates, no smoothing, and sparse-data follow-on acquisition. Its earlier argmax candidate is not independent observed authority.
• SPEC-fundamentals-build-2026-09-12.md: common model contract and distinction between a generic constructor and a machine instance.
• Source-pinned pattern applications and explicit relations are in cascade.json. Three new patterns are draft additions to the independent prediction sublibrary; no AIF arguments are copied into it.

WM-04 remains open. The deliverable here is a reviewable full build design with explicit stopping points, not a claim that the required observations or model now exist.

Explicit design relations

Existing AIF source snapshots

Quoted as local library references, not copied into the new sublibrary. Their other claims and examples are not adopted by this cascade.

@flexiarg aif/measurement-window-hygiene
@title Measurement-Window Hygiene (Tight Baseline, Settled Readback, Drop the Confounded)
@sigils [⊢⊣/≈]
@keywords measurement, baseline, settled read, transient spike, stale begin, drift, before-after, eventual consistency, prediction error, confound
@audience anyone measuring an action's effect on a drifting, settling system
@tone practical
@style pattern

@see-also aif/term-to-channel-traceability
@why no matching problem dossier node (@holds-at absent; not among the twenty dossier nodes R1-R17, R20, TRACE). Negative claim per the committed check futon2 holes labs library-loop runs l3_no_dossier_check.py, receipt runs L3-no-dossier-check.edn (enumerates every aif pattern's @holds-at against the dossier node set extracted from PROBLEMS-*.md headings). Rationale not invented (L3 rationale-backfill pilot, zai-1, 2026-09-05)
@how own mechanism: To measure an action's effect on a system that drifts and settles over time, capture the BASELINE tight — right before the action, not before the preparatory work — and read the EFFECT settled — after the state stabilizes (≥2 consecutive reads agree within ε) — and EXCLUDE any pair where drift or transience confounds the window. A before/after number is only as good as the window it brackets. (source: this pattern's ! conclusion line; L3 rationale-backfill pilot, zai-1, 2026-09-05)
! conclusion: To measure an action's effect on a system that drifts and settles over time, capture the BASELINE tight — right before the action, not before the preparatory work — and read the EFFECT settled — after the state stabilizes (≥2 consecutive reads agree within ε) — and EXCLUDE any pair where drift or transience confounds the window. A before/after number is only as good as the window it brackets.

  + context: You measure `effect = after − before` around an action, in a system whose state
    propagates asynchronously (scans, caches, eventual consistency, a live field other agents touch).

  + IF:
      You record a predicted/realised (or before/after) pair to score an action's effect on such a
      system.

  + HOWEVER:
      Two window failures inflate or fabricate the number, and both look clean:
      (a) STALE BASELINE — you captured "before" *early* (e.g. before minting the artifact the action
      commits), and the system drifted in between; the measured Δ is mostly that drift, not the action.
      (b) TRANSIENT READBACK — you read "after" at the first post-action sample, catching an in-flight
      spike that decays to baseline; the measured Δ is a scan-timing artifact.

  + THEN:
      Bracket the window tightly. BEFORE: capture the baseline immediately prior to the field-visible
      action — do the preparatory work first, *then* baseline, *then* commit. AFTER: read at the
      settled state — require ≥2 consecutive reads agreeing within ε; tag `:settled` vs `:transient`
      and exclude transient. And if the system drifted independently inside the window, DROP the pair
      (untagged / proposal-mode) rather than attribute the drift to the action.

  + BECAUSE:
      A measurement's validity lives in its window, not its arithmetic. The durable effect is what
      persists between a tight baseline and a settled readback; everything else — drift, spikes — is
      the window leaking. Excluding a confounded pair keeps the dataset honest; recording it launders
      noise as signal. (The two failures, caught live: a 0.277 "move" that was a transient spike
      (retracted on settled re-read); and a 0.277 "move" that was field drift during artifact-minting
      (excluded, proposal-mode) — same magnitude, both fake, both caught by tightening the window.)

  + how-to-apply:
      Order operations: prepare → baseline → act → settle → read → (exclude if confounded). Never
      baseline before the slow part; never read before the fast part settles; never attribute a
      window's drift to the action inside it. Distilled from the FutonZero G-SIM calibration arc
      (WM-pilot, 2026-06-11). Sibling of `[[aif/no-self-certification]]`.

@flexiarg aif/structured-observation-vector
@title Structured Observation Vector
@sigils [📎/了]
@audience fulab agents, aif implementers
@tone technical
@style pattern
@holds-at R2   ;; pegged 2026-08-30 (claude-15): the catalogue paragraph numbered R2 in p4ng/sec-catalog.tex; see PREREG-war-machine §1b

@how aif/predictive-coding-belief-update
@see-also aif/evidence-precision-registry
@why R2 dossier problem: R2 solves how heterogeneous evidence becomes comparable input to one control loop: the roster places **“Structured observation”** in PERCEIVE (`p4ng/empirics-futon/control-stages.edn:15`; control-stage numbering). (source: futon2 holes labs wm-contract PROBLEMS-r2-r8-r7-batch2.md, R2, The problem it solves; L3 rationale-backfill pilot, zai-1, 2026-09-05)
@how own mechanism: Treat each step as a typed observation vector so scoring and learning can be compared across time. (source: this pattern's ! conclusion line; L3 rationale-backfill pilot, zai-1, 2026-09-05)
@why problems/operator-turns-become-inference-observations (L14 edge-resolution; basis: holds -- the problem node's own holds-at token covers this pattern; source: receipt runs/L14-edge-resolution.edn; zai-1, 2026-09-05)
@why problems/r2-structured-observation (L14 edge-resolution; basis: holds -- the problem node's own holds-at token covers this pattern; source: receipt runs/L14-edge-resolution.edn; zai-1, 2026-09-05)
! conclusion: Treat each step as a typed observation vector so scoring and learning can be compared across time.

  + context: A pattern-guided agent must decide based on tool output, trace events, and user feedback.

  + IF:
    You want the agent's observation to be comparable across steps so scoring and learning do not collapse into prose interpretation.

  + HOWEVER:
    Tool output and user feedback arrive as heterogeneous artifacts (json, logs, diffs, snippets, natural language) that are hard to compare or weight consistently.

  + THEN:
    Maintain an explicit (o) as a typed, normalized feature map derived from session state (for example: test status, compile status, diff size, failing spec count, user-stated constraints, time since last anchor, contradiction flags).

  + BECAUSE:
    A stable observation vector is the prerequisite for meaningful precision control and policy scoring.

  + NEXT-STEPS:
    - Define :aif/observation-vector schema in tatami_schema.clj with keys [:test-status :compile-status :diff-size :failing-spec-count :user-constraints :time-since-anchor :contradiction-flags].
    - Add (extract-observation-vector session-state) to aif_bridge.clj that normalizes tool output into the typed map.
    - Include observation vector in :aif/tick events; require it for PSR context/anchors validation.
    - Add (observation-delta o1 o2) and log deltas alongside policy changes in learn_or_act/decide.

Download the source-pinned cascade JSON · WM-05 cascade