R12 two-layer calibration

Authored design — 2026-09-15. Section 3. This cascade elaborates the existing checklist item. It is not a dispatch, implementation receipt or completion verdict. Work outside WM-01–14 still requires Joe’s explicit greenlight; having a cascade is necessary but not permission.

Section 3/4 index · Exact TODO · Cascade JSON

Situation and design problem

Internal calibration can pass while the model fails to predict useful outcomes. R12 separates those claims and reconciles historical campaign evidence without rewriting it.

Governing checklist snapshot

R12 two-layer calibration. UNRESOLVED: reconcile the historical Campaign S claims (P121/P125/P130) with original records, and retain current producer/consumer plus independently grounded Layer 2 evidence. Layer 1 must remain labelled never-value-evidence. Catalogue R12 is not the differently numbered hyperparameter-inference contract. Do not silently omit this node because the recent DAG focuses elsewhere. Built evidence: Catalogue distinguishes internal and external calibration. Historical V7 census retained 892 records/69 R12 routes, but zero calibration-field/process-identity records; its seven checks and four controls pass only that census scope

This is the checklist’s dated evidence account, not a new deployment census. Existing source/test evidence keeps its bounded credit. The acceptance obligation above governs this design.

Pattern applications

R12-1 — Separate the two claims

Pattern: aif/two-layer-calibration

Declare L1 prediction-versus-model-realisation consistency as never-value-evidence, and L2 prediction versus independently witnessed outcome. Produce distinct estimands, domains and consumer claims; no combined pass flag.

R12-2 — Reconcile historical evidence

Pattern: wm-purpose-and-account/irrecoverable-history-stays-historical

Resolve P121/P125/P130 against original Campaign S records, keeping source version, process identity and missing fields. Produce supported, refuted or unavailable historical claims. Later demonstrations cannot recover absent originals.

R12-3 — Acquire independent L2 pairs

Pattern: aif/two-layer-calibration

Use authorized criteria and exact prediction/outcome joins from R2/WM-04. Retain independence and coverage; copied predictions, fallback values and transient observations cannot certify value.

R12-4 — Connect current calibration use

Pattern: wm-connected-learning/observation-to-next-choice

Trace the current calibration producer and its distinct consumer, with admitted revision and actual effect. Keep L1 diagnostics visible while only eligible L2 evidence supports the external outcome claim.

Composition and interfaces

Each node’s stated artifact is the input or constraint for the related node below. These are design dependencies, not a claim that a runtime scheduler automatically fires patterns. Existing WM producers are reused through the linked contracts; no second implementation is implied.

Retain the actual subject, source/model revision, task/occurrence, cutoff, producer and consumer for each interface. A missing field is a named gap; a default that changes the meaning is not an adapter. Reuse existing records where they satisfy these obligations. If a new component is indispensable, identify its exact TODO/cascade clause before seeking a separately bounded implementation instruction.

Worked situation

A constant predictor matches a model-generated realised score. L1 may be consistent, but a receiver still cannot use the handoff. Only an independent outcome criterion can reveal that failure at L2.

This is a distinguishing design example, not a claimed production experiment.

Implementation sequence and closure evidence

1. Bind the first application to the actual source, task and authority. Resolve the decisions below before their dependent change; do not reproduce a known gap merely to claim activity.
2. Implement only the specifically dispatched producer/consumer change from the middle applications. Retain the meaningful pre-change failure and relevant post-change controls; do not start a general infrastructure project.
3. Acquire the final application’s evidence and independent review. Compare it against every clause of the exact TODO, identifying any still-open dependency. Source, tests, serving state, ordinary use and whole-run validation receive different credit.

Distinguishing controls: Reject predicted-as-realised, no process identity, cross-occurrence pair, absent labels counted as outcomes and L1 promoted to L2. Do not treat 892 historical records as 892 eligible pairs.

Closure rule: every part of the quoted obligation must be supported by the indicated artifacts and actual connections. A useful rejection proves the rejecting boundary only. One accepted subclause does not tick the whole item. A bellback must supply the exact reviewed subject, executed checks, independent verdict and remaining clauses.

Dependencies and authority boundaries

R2 · R4 · R15 · E07

The catalogue R12 is not a differently numbered hyperparameter contract. Resolve historical contradictions explicitly; do not omit R12 from current scope.

Dependency links describe required interfaces; they do not authorize work on those items. No implementation outside the existing TODO list is approved. Publication does not select a numerical law, settle an unresolved operator ruling or authorize a run.

Pattern and source scope

All seeded patterns are applied above. The JSON pins their source bytes and the governing checklist/rulings. Historical formulas, thresholds and examples in a pattern remain scoped examples unless the governing contract adopts them. This draft adds no pattern-library entries and claims no independent semantic ratification.

Existing pattern source snapshots

Reference text; historical examples and prescriptions retain the scope stated above.

@flexiarg aif/two-layer-calibration
@title Two-Layer Calibration (Dynamics-Consistency vs Witnessed-Outcome)
@sigils [L1/L2]
@keywords calibration, prior-equals-value, dynamics, value, witnessed outcome, three-witness, self-reference, G-vs-G, reward gating
@audience anyone calibrating a value/forward model against a field the model helps compose
@tone architectural
@style pattern
@holds-at R12   ;; pegged 2026-08-30 (claude-15): the catalogue paragraph numbered R12 in p4ng/sec-catalog.tex; see PREREG-war-machine §1b

@how aif/off-continuity-null-discriminates
@why-posthoc aif/no-self-certification
@why R12 dossier problem: Control-stage R12 solves separating internal consistency from externally witnessed value evidence: the roster places **“Two-layer calibration”** in BELIEVE's assurance band (`control-stages.edn:40`; control-stage/catalogue numbering). (source: futon2 holes labs wm-contract PROBLEMS-assurance-band-batch6.md, R12, The problem it solves; L3 rationale-backfill pilot, zai-1, 2026-09-05)
@how own mechanism: When you calibrate a model whose own outputs compose the field it is scored against, split the calibration into TWO layers and never let one stand in for the other. L1 — dynamics/consistency (does the field respond to measured state?): cheap, accrues every cycle, and is NEVER value evidence. L2 — outcome-vs-prediction (did real functioning result?): scarce, witnessed, gated, and is the ONLY value clearance. (source: this pattern's ! conclusion line; L3 rationale-backfill pilot, zai-1, 2026-09-05)
@why problems/r12-two-layer-calibration (L14 edge-resolution; basis: holds -- the problem node's own holds-at token covers this pattern; source: receipt runs/L14-edge-resolution.edn; zai-1, 2026-09-05)
! conclusion: When you calibrate a model whose own outputs compose the field it is scored against, split the calibration into TWO layers and never let one stand in for the other. L1 — dynamics/consistency (does the field respond to measured state?): cheap, accrues every cycle, and is NEVER value evidence. L2 — outcome-vs-prediction (did real functioning result?): scarce, witnessed, gated, and is the ONLY value clearance.

  + context: You are calibrating a forward/value model against a live field, and the field's value
    function is computed from the same effects the model predicts with (`prior == value` risk).

  + IF:
      Predicted-vs-realised pairs are your evidence that the model is "calibrated".

  + HOWEVER:
      On such a field the comparison is largely self-referential: it tests dynamics-consistency
      (did the world move as the model's state-response said?), NOT whether the model's VALUE is
      right — because the realised IS the model's own output. A model can pass this layer forever
      while being worthless at predicting real outcomes. Worse, fixing the obvious circularity
      (making predict-effects state-sensitive) RELOCATES it rather than removing it: state-blindness
      breaks, but the error *magnitude* stays model-priced.

  + THEN:
      Name and keep both layers explicitly. **L1 = G-vs-G dynamics**: cheap, every cycle, labelled
      never-value-evidence; it earns its keep as a consistency/sanity layer. **L2 =
      outcome-vs-prediction** against WITNESSED outcomes the model did not produce — closure-folds,
      three-witness certificates, attested real functioning — scarce, gated behind the reward
      discipline. Only L2 clears value. Forbid L1's cheap pass from being reported as L2 clearance.

  + BECAUSE:
      The only thing that escapes `prior == value` is a referent the model had no hand in. L1's
      referent (the field) is the model's own output, so L1 can only ever measure internal
      consistency. L2's referent (a witnessed outcome) is external, so it is the only layer that can
      be WRONG about value — which is exactly why it is the gate, not an enhancement.

  + canonical-instance:
      FutonZero G-SIM (WM-pilot, 2026-06-11): L1 dynamics live — the state-sensitive model is 2.55×
      better than the constant counterfactual at predicting no-move — while L2 stays gated behind
      pudding-G1's binding of the arrow-witness to witnessed affect/outcome. The split was forced by
      discovering the calibration was self-certifying (`[[aif/no-self-certification]]`).