# SAE-1.4 — VALIDATION SUMMARY
### What has been tested, what passed, what FAILED, and the honest scope

> **PURPOSE:** This document closes the gap previously flagged ("the paper says validation
> remains to be developed"). That statement in the paper (v0.6) is STALE — it was written
> before this work was done. Validation exists. This file is the canonical record of it.
> The runnable artifacts live in `SAE14_analysis/`. Every number below was independently
> reproduced from the committed code (deterministic, seed=42).

---

## TL;DR

SAE-1.4 has been structurally tested (passes) and empirically pre-validated on honest
synthetic data (beats baselines out-of-sample, with one pre-registered hypothesis that
failed — shown openly). It has NOT been validated against real human data or real
consciousness. The honest status is: a formally specified, internally tested, empirically
testable directional model — not a proven description of consciousness.

This means the skeptic-conversion / "Evidence room" path can be built honestly now, because
the failure on display is a real failure we actually found — not aspiration.

---

## 1. Structural testing — PASSED

- 5,000 randomized property trials over the full input space.
- All invariants in `SAE-1.4-CANONICAL-SPEC.md` §10 hold: bounded inputs stay bounded;
  constraints never increase effective Shakti; in the aligned regime (Belief_total > 0) Purity
  raises access and Blockers lower it, monotonically; stage 0/1 never defaults to 0.
- Artifact: `SAE14_analysis/test_structural.py`
- **What this establishes:** the implementation behaves according to its own rules.
- **What this does NOT establish:** that the rules describe reality. (State this explicitly
  anywhere results are shown. "Passed structural tests" ≠ "validated.")

---

## 2. Empirical pre-validation on synthetic data — PASSED, with honest caveat

### Methodology (the part that makes it non-circular)

The synthetic ground-truth world was deliberately built with a DIFFERENT functional form than
SAE-1.4 — saturating (tanh) purity response, multiplicative blockers, a smooth belief gate,
diminishing-returns Shakti, heavy-tailed additive grace. SAE-1.4's additive bracket is only an
approximation of that world. This avoids circularity: SAE-1.4 was NOT tested against data it
generated. It had to compete to recover an independent causal structure out-of-sample.

Critically, the ground truth was built so physical constraint does NOT reduce inner awareness
(only an expression channel) — to test the fairness invariant as a trap.

- Dataset: 200 synthetic users × 120 days = 24,000 rows. Artifacts: `SAE14_analysis/gen_data.py`,
  `hope_synth.csv`.
- Analysis: `SAE14_analysis/analyze.py`. Captured output: `results_output.txt`.

### Results

| Finding | Result |
|---|---|
| Predictive (walk-forward, out-of-sample) | SAE-1.4 components model R² ≈ 0.896 vs best baseline ≈ 0.156. Earns its keep on data it did not generate. (Fixed-form raw composite ≈ 0.584.) |
| Belief-regime stratification | corr(Purity, Awareness) = +0.489 in positive-belief regime vs +0.403 pooled. Confirms you MUST stratify on belief regime or misread results — exactly as the gate structure predicts. |
| Directionality (cross-lag) | Blockers(t) → Awareness(t+1): β = −6.19, p ≈ 3e-11 (strong). Reverse (Awareness(t) → Blockers(t+1)): null, p ≈ 0.93. Arrow points as the model claims. |
| Fairness check | After matching on Purity + Belief, high- vs low-constraint users differ by only +2.70 on a 0–100 scale. Small residual leak (mental-constraint/blocker confound), within fair range. |

### The FALSIFICATION finding — show this openly (it is the trust signal)

- **Pre-registered hypothesis:** "Purity(t) reduces future Blockers(t+1)."
- **Result:** did NOT hold (β ≈ 0, p ≈ 0.91).

This pathway was not built into the ground truth, so the test correctly found nothing. The point:
SAE-1.4 is genuinely falsifiable, and here is a sub-claim that failed. If real HOPE data also
shows this null, the "purity dissolves future blockers" claim must be dropped or rebuilt. This
is the first thing to test against real logs. Publishing this failure is the single strongest
anti-cult, pro-credibility move available — do not hide it.

---

## 3. Honest scope (must travel with any presentation of these results)

- **Validates:** structure, internal coherence, falsifiability, baseline-beating on synthetic data.
- **Does NOT validate:** that SAE-1.4 describes real consciousness. The data is synthetic.
- **Real validation requires:** real, anonymized HOPE logs, ≥ 60–120 days, with a daily
  ObservedAwareness proxy (e.g. a 0–100 Witness-access / clarity / steadiness self-rating).

---

## 4. How this maps to the experience (for the Library "Evidence" room)

The Evidence room can truthfully show, in Phase 1:

- The structural-test pass (with the "≠ validation" caveat).
- The synthetic out-of-sample result (R² 0.90 vs 0.16) — clearly labeled synthetic. Show both the
  components model (0.896, "right variables?") and the fixed-form raw composite (0.584, "right
  functional form?") so the claim is precise.
- The null result, prominently — failure on display.
- The downloadable code (`SAE14_analysis/`).
- The open problem: "needs real anonymized data — come help" (recruitment hook).

What it must NOT do: present synthetic validation as validation against real consciousness, or
imply the model is "proven." The confidence and the caveat appear together, always.

---

## Reproduction note (independent re-run)

The full suite was re-run from a clean environment: structural tests PASS (all five invariants,
5,000 trials); `gen_data.py` (seed=42) regenerates `hope_synth.csv` numerically identical to the
committed file (max abs diff ~3e-14); `analyze.py` reproduces every number in `results_output.txt`
to the printed digit.
