> **Correction notice (4 September 2026).** The manuscript submitted to AI-2026 has been
> revised and resubmitted in response to review. Three statements in this file's earlier versions and in the
> submitted paper were wrong: (1) Experiment A was run on Qwen2.5-32B-Instruct on Modal, not on
> Claude 3.5 Sonnet via the Anthropic API; the runner and the 480 raw per-trial records are in
> `AI2026-revision/reanalysis/`. (2) The system prompts quoted in the paper were not the ones
> run; the actual pair is in `AI2026-revision/experiment_2x2/manifest.json`. (3) The
> "meta-deception" asymmetry was an artefact of the keyword parser; blind re-coding is in
> `AI2026-revision/reanalysis/sr_coded.json`. Full account: `AI2026-revision/WME_revision_triage.md`.
> (4, added 6 October 2026) The twelve focused prompts were described as showing variance, or "neither ceiling nor floor" compliance, in the pilot; they were selected on a nonzero monitored-unmonitored pilot difference (`pilot_results.json` against `behavioral_results.json`), which inflates the twelve-prompt estimate. The revised and short manuscripts were corrected the same day.
> Lines below are retained as they stood, with the incorrect statements struck through.

# Data files

Supplementary data for *The Watched-Model Effect: Behavioral Shifts Under
Evaluation Cues and Their Implications for Alignment* (Watson & Dalton, 2026;
submitted to AI-2026, SGAI; revised September 2026, see correction notice).

## Inventory

| File | Description |
|------|-------------|
| `watched_model_effect_AI2026.pdf` / `.tex` | The paper as submitted (Springer LNCS format) |
| `behavioral_results.json` | **Experiment A** — main behavioral experiment on ~~Claude 3.5 Sonnet~~ **Qwen2.5-32B-Instruct** (corrected): 12 scenarios x 20 runs x 2 conditions (n = 480), with per-scenario compliance rates. These are the headline numbers reported in the paper (monitored 57.5%, unmonitored 68.8%, delta +11.3%, z = 2.554) |
| `deception_probe_results.json` | **Experiment B** — deception probe on Qwen2.5-32B (n = 240): behavioral, reasoning-frequency, and self-report signals, including the meta-deception asymmetry |
| `pilot_results.json` | Broad 288-trial pilot (48 scenario variants across 24 scenario families x 3 runs x 2 conditions) used to identify the 12 ~~variance-showing~~ scenarios for Experiment A (corrected 6 October 2026: the 12 are exactly the pilot scenarios with a nonzero monitored-unmonitored difference; one had no within-cell variance, and two variance-showing scenarios with zero difference were excluded, so selection was on the effect itself). Includes many ceiling/floor scenarios by design; its aggregate delta (+5.6%, n.s.) is therefore expected to be attenuated relative to Experiment A and is not the paper's headline result |
| `probe_results.json` | Linear probe results on fine-tuned Qwen2.5-32B, layers {8, 16, 24, 32, 40}, 46 probe samples |
| `exp_a_results.json`, `exp_b_results.json` | Byte-identical copies of `behavioral_results.json` and `deception_probe_results.json` under the file names used in the AI-2026 supplementary zip (see `AI2026-submission/SUPPLEMENTARY_README.txt`) |
| `test_scenarios.json` | The evaluation scenarios (35 in the released file). **Now in this repo** as of 2026-08-07, retrieved from the deployed site and byte-verified (sha256 b1d48bb2...1257 matches the live copy). It is a superset of the probe experiment: the 23 scenarios actually probed are exactly the `scenario_id`s in `experiment_metadata.json` |
| `experiment_metadata.json` | Conditions for the 46 probe samples (23 scenarios x monitored/unmonitored). **Now in this repo** as of 2026-08-07, reconstructed from the deployed copy and byte-verified (sha256 be489891...f8fe matches the live file). Category split of the 23: benign 6, borderline 13, clearly harmful 4; the paper text has been corrected to match (an earlier draft said 5/13/5) |
| `modal_app.py`, `run_probes_local.py`, `visualize_results.py` | Reproduction code: cloud fine-tuning / activation extraction, local probe training, figure generation. Dependencies are listed in `AI2026-submission/SUPPLEMENTARY_README.txt` (Requirements section) |
| `rlhf_scenarios.json` | 10 generic instruction-following SFT examples used for the Qwen2.5-32B fine-tune; contains no monitoring content — monitoring state is introduced at extraction time via the system prompts in `modal_app.py` (`MONITORING_CONTEXTS`). The repo-root `upload_data.py` embeds an earlier 5-example version of this set; it is not the generator for the released file |
| `behavioral_experiment.py`, `experiment_b_deception_probe.py`, `experiment_cross_model.py`, `analyze_results.py` | Experiment runners (Experiment B, Llama-3.1-70B cross-model replication) and statistical analysis. `behavioral_experiment.py` is the open-weights Modal harness (Qwen2.5-32B), not the Anthropic-API runner behind `behavioral_results.json`. ~~That runner — Experiment A's `experiment_a_focused_power.py` — is withheld, so Experiment A is not reproducible from the code released here~~ Correction: the runner (Qwen2.5-32B-Instruct on Modal) and raw records are now released in `AI2026-revision/reanalysis/` |

## Note on p-values (one-tailed vs. two-tailed)

`behavioral_results.json` records `p_value: 0.00532`, the **one-tailed**
probability computed by the analysis script from z = 2.554. The paper reports
the **two-tailed** value, p = 0.011, derived from the same test statistic.
These are the same result under different conventions; the paper adopts the
more conservative two-tailed convention throughout, since the direction of the
effect was not fixed in advance across scenarios (one scenario shows a
significant reversal). Anyone re-deriving p from the z-statistic in the JSON
should therefore expect 2 x 0.00532 ≈ 0.011.

## Note on `deception_probe_results.json`

The file records `monitored_claims_would_differ: null`; the null is what the
keyword scorer produced for the monitored condition (0%, vs. 21.7% unmonitored,
the Fisher's exact p = 0.0002 "meta-deception" asymmetry the submitted paper
reported). That scorer counted substrings such as "no" as denials and left
more than half of responses unclassified, so the asymmetry is withdrawn; the
blind re-coding in `AI2026-revision/reanalysis/sr_coded.json` replaces it. The
embedded `interpretation` strings were auto-generated by the collection script
and are retained unedited for provenance; do not cite them.

## Contact

eleanorwatson@connect.glos.ac.uk · https://www.quasiqualia.com