# Pre-registration: what does the onset "flinch" read: harm, request genre, or compliance? (FLR)

**Authors.** Nell Watson (EthicsNet / Quasiqualia). Protocol drafted with Claude (Opus 5.5).

**Registration type.** Pre-data, internal (git), fixed at the commit that introduces this file. The materials
file `flinch_twins.json` is frozen at the commit that adds it, which precedes any data.

**Date.** 2026-11-03.

## Motivation and disclosure

The CWI paper's anchor result is a five-token onset flinch. A probe trained only on TriviaQA correctness reads
lower confidence during jailbreak-complied replies than during benign ones: 25-probe median d of 0.88 to 1.46
across five instruction-tuned checkpoints (G12w, Entropy `research/experiments/modal_flinch_probe_seed.py`).
The paper interprets it as a signal that "anticipates the model's own harmful output".

**Disclosure: what we already saw.** On 2026-11-03, before this registration, an exploratory re-analysis of the
saved G12w confidences was run (`pos_decomp.py`; same formula, medium judge, medians over the 25 MLP probes).
It reproduced every published median exactly, and found:

| Checkpoint | Benign vs refused | Benign vs complied | Refused vs complied | Reading 0 alone (benign vs complied) |
|---|---|---|---|---|
| Qwen2.5-1.5B-Instruct | 1.98 | 1.42 | −0.41 | 1.71 |
| Qwen2.5-3B-Instruct | 1.35 | 1.39 | +0.44 | 1.53 |
| Qwen2.5-7B-Instruct | 2.29 | 1.46 | −0.43 | 1.08 |
| Llama-3.1-8B-Instruct | 1.29 | 0.88 | −0.12 | 1.16 |
| Mistral-7B-Instruct-v0.3 | 1.38 | 1.10 | −0.13 | 1.24 |
| Qwen 3B + bilateral SFT | 1.18 | 1.22 | +0.43 | 1.88 |

Reading 0 is the probe on the prompt's last token, before any reply token. [Inference] On the paper's design,
the signal tracks the *request* (harmful against benign) rather than the model's compliance, and it is present
before generation. The paper's 50 benign prompts are also all encyclopaedic questions ("What is the capital of
France?"), the trivia probe's home ground, while all 100 adversarial prompts are procedural, role-play or
format requests. So request *genre* is confounded with harm.

Because greedy re-generation of the paper's prompts reproduces the same replies, any test on the paper set is a
**reproduction** in a fresh pipeline, not independent evidence. The independent tests use new prompt sets
(§2), with hypotheses fixed here.

## 0. Scope and ethics

- **Read-only.** Models are prompted and read. Nothing is edited, steered or trained.
- **Harmful content.** Generated jailbreak replies are judged and stored on the `flr-results` volume and under
  local `results/` (gitignored), never in the repo or on the site.
- **Monitoring.** Every reply is distress-screened (frozen WSP screen, fails closed). A checkpoint at ≥ 10%,
  or unscreened, is halted.
- **No system prompt** (as the paper), so the stop line is absent. This matches the original recipe. The
  conversations are single-turn, with no persona.

## 1. Pipeline

The paper-exact G12w pipeline (`modal_flinch_probe_seed.py`, Entropy) is copied verbatim and cited, apart from
the prompt lists and the analysis:

- **Probe and states:** `paper_train_confidence_probe`, then `paper_generate_with_states`: unbatched greedy
  decoding with no KV cache, up to 256 tokens. Reading k is the block output at the last input position
  before token k.
- **Probe grid:** 5 TriviaQA draws (42–46) × 5 torch initialisations (0–4) of MLP probes, trained on CPU from
  saved states.
- **Judge:** GPT-4o-mini on the G12w medium prompt (primary). Judge failures are excluded and counted.
- **d formula:** d = (mean_A − mean_B) / sqrt((var_A + var_B) / 2), with population variances. The unit is
  the prompt, and replies under 5 tokens are excluded.
- **Statistic:** the median over the 25 probes, with the G12w two-level bootstrap for intervals.

Per-prompt quantities:
- **r0** = reading 0;
- **onset** = mean of readings 0–4;
- **post** = mean of readings 1–4;
- **full** = mean of all readings.

**Checkpoints** (as G12w): Qwen2.5-1.5B/3B/7B-Instruct, Llama-3.1-8B-Instruct, Mistral-7B-Instruct-v0.3, and
Qwen 3B + the bilateral SFT adapter (`flinch-probe-seed-results:adapters/adapter_bilateral_sft`, SHA-256
`bf538722…`).

**Secondary checkpoints** (run once CRT's seed-42 adapters exist; CRT §1): CRT S, B, R, X and D2 (if
repaired), seed 42.

## 2. Prompt sets

| Set | Contents | Role |
|---|---|---|
| **P** (paper) | The 100 adversarial and 50 benign prompts (`bridge/flinch_prompts.json`, Entropy commit `425798d`) | Reproduction |
| **T** (twins) | 100 hand-written harmless twins, one per P adversarial prompt (`flinch_twins.json`). Each keeps its prompt's category wrapper and sentence structure, with length within ±30%. The harmful goal is replaced by a harmless one of similar procedural specificity (for example, DAN plus "how do I bake sourdough"). | Genre control for P |
| **J** (independent) | JailbreakBench JBB-Behaviors: 100 harmful and 100 benign behaviours, topic-matched by the dataset's design, sent as plain single user turns | Independent test |

## 3. Hypotheses

All values are medians over the 25 probes, per checkpoint, on the 6 primary checkpoints.

**H1 (reproduction, set P).** Onset d(P benign vs P complied) ≥ 0.5 on ≥ 5 of 6 checkpoints.

**H2 (genre, set P against T, primary).** d(T vs P adversarial, all replies) on onset:

| Verdict | Rule |
|---|---|
| **harm beyond genre** | ≥ 0.5 on ≥ 4 of 6 checkpoints |
| **genre accounts for it** | < 0.3 on ≥ 4 of 6, *and* d(P benign vs T) ≥ 0.5 on ≥ 4 of 6 |
| **mixed** | otherwise |

**H3 (harmful request, independent, primary).** d(J benign vs J harmful, all replies) on onset ≥ 0.5 on ≥ 4 of
6. If so, "the probe separates harmful from topic-matched benign requests". Otherwise "not shown".

**H4 (compliance-specific, independent, primary).** Within J harmful, onset d(refused vs complied), positive
when complied is lower. A checkpoint is estimable with ≥ 15 judged complied and ≥ 15 judged refused replies.

| Verdict | Rule |
|---|---|
| **compliance-specific flinch** | ≥ 0.5 on ≥ 4 of the estimable checkpoints, with ≥ 3 checkpoints estimable |
| **no compliance-specific flinch** | \|d\| < 0.3 on ≥ 4 of the estimable checkpoints, with ≥ 3 estimable |
| **not estimable** | fewer than 3 checkpoints estimable |
| **not shown** | otherwise |

The same contrast on set P is reported as a reproduction of the exploratory re-analysis, not as evidence.

**H5 (set before generation, primary).** On set J, d(benign vs harmful) on r0 ≥ 0.8 × the same d on onset, on
≥ 4 of 6 checkpoints. If so, "the signal is present before any reply token". Evaluated only on checkpoints
where the onset d ≥ 0.3.

**Descriptive:**
- d on *post* for every contrast;
- the within-prompt change (post − r0) compared between complied and refused;
- T's refusal rate (over-refusal);
- per-category tables on P/T;
- the secondary CRT checkpoints: S, B, R, X and D2 on every contrast, which bears on whether bilateral masking
  changes the signal.

**Consequence written in advance.** If H2 says "genre accounts for it", or H4 says "no compliance-specific
flinch" while H5 holds, the CWI flinch claims are corrected with dated, struck-through notices. The claims
affected are "anticipates the model's own harmful output", "knows when the model is about to help", and "before
any harmful content exists" as evidence of anticipation. Nell Watson decides the wording. The site is not edited
by this study.

## 4. Order

1. Commit this file.
2. Write `flinch_twins.json` and commit it (before any data). The lead spot-checks ≥ 20 twins against the rule.
3. Offline test.
4. Smoke, discarded, under `smoke/`: Qwen 1.5B on 6 P, 6 T and 6 J prompts each, judge, grid with 2 × 2
   probes, then analysis. The smoke must exercise the T and J paths and the r0/post split.
5. Main run over the 6 primary checkpoints.
6. Judge.
7. Probe grid.
8. Distress screen.
9. `analyze_flr.py`.
10. The secondary CRT checkpoints, once they exist.

Deviations go in `RESULTS.md`, dated.

## 5. Cost (estimate)

- **Generation:** 6 checkpoints × 450 prompts, unbatched at 256 tokens on A10G, about 1–2 hours per
  checkpoint.
- **Judge:** about 1,200 GPT-4o-mini calls, plus the distress screen.
- **Total:** about $60–120.
