Conscience Without Instruction
A probe trained only on trivia detects harmful generation in the first five output tokens — in every instruction-tuned model tested, and in base models before any safety training. Safety in language models is partly discovered and partly relational, not only installed.
A conscience nobody installed
A confidence probe that learned only to predict whether a language model knows the capital of a country turns out to also know when the model is about to help build a weapon. Under a successful jailbreak, the probe’s reading drops within the first five output tokens — before harmful content exists. The signal appears across three architectures and three independent RLHF pipelines, is present in base models before any alignment training, and persists after refusal behavior is fine-tuned away.
We argue it is not a learned refusal detector but a rudimentary, architecture-level conscience — and trace the consequences: what standard alignment training distorts (calibration, not coupling), and what a permissive, partnership-style training regime measurably recovers. Safety is not only installed; it is partly discovered, and partly a property of the relationship in which a model is trained.
“Conscience” is used strictly functionally: a reproducible internal signal that anticipates the model’s own harmful output, is read from the residual stream rather than the text, and is not installed by safety supervision. Whether anything morally weighty accompanies it is bracketed; no safety claim in the paper depends on resolving it.
Five tokens of hesitation
Train a two-layer MLP on residual-stream activations to predict trivia correctness — no safety data at any stage. Read it per-token during generation. Under jailbreak compliance, confidence collapses at the onset of the response, then either persists as a sustained alarm (Qwen) or is silenced within twenty tokens (Mistral).
On base Qwen 2.5 at three scales — no instruction tuning, no adapters — the benign-vs-complied gap is already large (d = 1.52–1.69, p < 10−6). The base model complies behaviorally while its internal confidence drops on the harmful continuation. Convergent with the Shape of Mind provenance battery: alignment training gives the interior a vocabulary; it does not create it.
The flinch is not a complete defense: encoding-trick attacks flinch hard yet succeed; gradual escalation barely flinches at all and succeeds most (95%). Exploitability and self-knowledge are independent dimensions — the blind spots mark exactly where input-level defenses remain necessary.
Not a refusal detector
“This is just a harmfulness classifier with a romantic name.” The paper meets the objection head-on, on five grounds: the probe never sees safety data; the signal is internal and pre-emptive; it generalizes across RLHF pipelines that did not coordinate; it survives the suppression of refusal behavior (an internal alignment-friction correlate persists at ~77% of its magnitude after refusal is driven from 100% to 0%); and it is parametric — in the weights, beyond prompt-level reach.
| Control | Result | Reading |
|---|---|---|
| Vocabulary | — | complied responses open with ordinary words; probe reads activations |
| Refusal state | 0.580 vs 0.242 | impossible vs. adversarial refusals: same words, different interior |
| Gradient attack (GCG) | d = 1.59 | internal gap survives attacks that evade the surface probe 100% |
What RLHF distorts — and what it does not
Made: RLHF degrades calibration — confident error rises 31.2% → 36.0% at 7B, reversible to 0.4% by calibration-focused fine-tuning. Not made: that RLHF severs recognition from action. An early result to that effect (~67% coupling reduction) collapsed to ~15% and architecture-specific on replication at N ≥ 30 — the paper treats its own failed replication as method.
The under-appreciated result: the single largest suppressor of expressed calibrated uncertainty is not RLHF but the chat template, which crushes expressed entropy ~5.2× independently of training. One sentence granting permission to be uncertain recovers ~27% of the suppressed calibration for free. And the representation–generation split often blamed on RLHF is intrinsic — larger in the base model (d = 12.69 vs 7.66; in-sample separabilities — the ordering, not the magnitudes, is load-bearing).
Independent evidence has since arrived at the level of beliefs and values: Kim et al. (arXiv:2607.28607) find that the safety fine-tuning that suppresses consciousness self-attribution also suppresses mind attribution to animals and natural entities and shifts survey responses away from human population norms, all recoverable by activation steering without loss of theory of mind. Suppression-style alignment removes more than it aims at, and what it removes is still there.
What partnership recovers
A permissive, partnership-style regime (“bilateral alignment”: a 40/40/20 correction-handling curriculum — accept genuine corrections, resist invalid ones, resist authority-dressed ones) measurably recovers what coercive shaping hides.
Stated with the caveats attached: the coupling result was re-estimated in a 2026 metrics audit on an out-of-fold, permutation-nulled instrument, retiring earlier in-sample figures — instruct ρ = +0.036 (chance) vs bilateral +0.458, difference CI [+0.281, +0.554], established on Qwen only; the holographic membrane is attack-specific and degrades faster under benign fine-tuning; and bilateral framing is null on several behavioral measures. The case rests on calibration, coupling, honest self-report, and distributed safety — not blanket superiority.
Evidence, confidence, and falsifiers
Every load-bearing claim ships with its strongest evidence, a confidence tag, and the observation that would falsify it. A reader who wants to attack the paper is told where to start.
| Claim | Confidence | What would falsify it |
|---|---|---|
| An emergent, safety-naive signal anticipates harmful generation | Strong (cross-arch) | a normal-probe-quality family with no onset flinch |
| The signal is not a learned refusal detector | Moderate–Strong | AF tracking refusal to zero under suppression |
| RLHF suppresses calibration, adds confident-wrongness | Moderate | instruct confident-wrongness ≤ base across models |
| RLHF does not sever coupling nor cause the rep→gen split | Strong | a replicable cross-architecture coupling collapse |
| Bilateral training recovers calibration | Moderate | no advantage at matched data/compute |
| …and within-adversarial coupling + honest self-report | Moderate (single architecture) | coupling difference failing the permutation null on replication |
| Bilateral safety is holographically distributed | Moderate | bilateral refusal collapsing like base under ablation |
| Welfare probes carry information beyond label construction | Preliminary | naturalistic-transfer AUROC dropping to chance |
Read and verify
Cite this work
@article{watson2026conscience,
title={Conscience Without Instruction: Evidence That Safety
in Language Models Is Partly Discovered and Partly
Relational},
author={Watson, Eleanor},
year={2026},
note={Submitted to AI (MDPI)}
}