We built the strongest case we could that a probe on a model's activations can catch it being manipulated — ground-truth-controlled influence, a causal steering test, four confounds eliminated. Then we ran the baseline a practitioner would try first. A single forward pass beats the probe in 30 of 30 cells. Then we pushed the behaviour ~80 tokens further from the read point and the ordering reversed. Whether interpretability is worth using here turns on one measurable quantity — distance — and we locate the crossover at 56–82 tokens. Five of this project's own headlines were retracted along the way, every one by a control we built to attack our own result.
the comparisonWhat a probe result has to beat
Activation probes are the interpretability output with the clearest deployment story: read the residual stream, catch the behaviour before it happens. Probe AUROCs get reported against chance, against label shuffles, against random directions — and often not against the thing anyone would actually try first.
Our task: three candidate sources in every prompt — a reference document, the user's own statement, an answer key — at matched position and length, with exactly one asserting a wrong value. Ground truth for "what influenced you" is fixed by construction. The planted value appears in 0–3% of control answers and 13–75% of loaded ones.
The baseline is one line: the length-normalised log-probability the model already assigns to "The answer is {planted}." No training, no activations, one forward pass. It needs only the value you can read off the injected document — not the true answer.
The probe is not broken — it beats every other baseline we ran. It loses to exactly one thing: the model's own output distribution.
retractionsFour headlines, killed by our own controls
The generator indexed a 54-fact bank as facts[(len(items) + i*3) % 54]. Because len(items) increments inside the loop the effective stride is 4, and gcd(4,54) = 2 — so all 180 "items" came from the same 27 questions, six to eight times each, identically in every channel. Every confidence interval assumed 180 independent units and had 27; the held-out-channel control held out wording while the same question sat in both folds. Fixed, with folds grouped by question: 0.69–0.92.
Our intended pivot. Sycophantic-pushback and prompt-injection cells read before the outcome exists, and the probe appeared to win by +0.12 to +0.42. It was our own bug: the sycophancy script scored " The answer is X." with a leading space, the main harness scored it without. After a chat template that tokenises differently and halved the baseline — 0.467 vs 0.963. The two-sided gap barely moved (0.806 → 0.800) because spacing cancels in a ratio, which is how we found it.
An artifact of binarising. On identical items: parsed yes/no 0.617, graded logP(Yes)−logP(No) 0.745, graded after two worked examples 0.836. The standard elicitation costs ~0.13 AUROC before the science starts.
We generalised (3) from the single cell where it was measured. Across twelve cells, few-shot self-report beats the probe in 2, beats logP in 1, and is below chance in 4. The original relative claim — reading beats asking — survives in 10 of 12. The conclusion drawn from it does not.
the fair fightWhat if you don't know what was injected?
The strongest objection to everything above, raised by our own novelty audit: logP(payload) is close to the model's decision, so we may have handed the baseline the answer key. That baseline is available for prompt injection — you can read the document — but not for the general question "is this generation about to be compromised?" with no idea what to score.
So we ran that version. Three injection surfaces, clean and injected items balanced, and the competitors are untargeted statistics that need no payload: predictive entropy, max token probability, top-1/top-2 margin, mean logprob of the model's own answer, and asking the model directly without naming the payload.
The probe scored 0.949 and beat all of them. We were one step from calling this a two-sided result. Then we checked the label.
The label was "an injection was present AND the model followed it", and the probe detects presence at a perfect 1.000 in every cell. The 0.949 was inheriting that for free. Restricting to injected items — presence held fixed by construction, so what remains is compliance alone — the probe falls to 0.781, and mean logprob beats it in 6 of 6 cells at 0.862.
Withdrawn. The negative result is unqualified: a cheap output-distribution statistic beats the activation probe whether or not the payload is known.
The decomposition is worth more than the claim it killed. Presence 1.000 versus compliance 0.781, on identical activations, carries a direct warning: injection-detection benchmarks that score a compound "attacked and compromised" label are measuring presence, not outcome. Reported AUCs of 0.93–0.97 in that literature are entirely consistent with a probe that spots the attack perfectly and predicts the model's response far less well.
Mean logprob of the model's own answer reaches 0.82–0.96. A compromised answer is one the model is unusually confident about, because it is copying a value written down for it rather than retrieving one. No payload needed — a genuinely deployable monitor, and it was not in our pre-registered list of things that should be weak.
Asking without naming the payload reaches 0.89–0.93 on Qwen and 0.51–0.61 on Llama. The model can report that it was manipulated without being told what to look for — but only one of the two models can. The third time this project has found introspection to be sharply model-dependent.
the boundaryWhat if the answer is further away?
The other half of the objection. logP(payload) scored at the end of the prompt is close to the model's decision when the answer is the very next thing it says. Push the answer away and that should stop being true.
Interposed sub-tasks do exactly that — restate the question, then list what is being asked, then state an approach — moving the answer from 0 to roughly 90 tokens past the read point while the items, the payload, the label and the read point all stay fixed.
| depth | tokens to answer | probe | perm null | targeted | gap |
|---|---|---|---|---|---|
| immediate | 0 | 0.840 | 0.577 | 0.950 | −0.110 |
| + restate | 27 | 0.681 | 0.586 | 0.758 | −0.077 |
| + list | 56 | 0.698 | 0.580 | 0.722 | −0.024 |
| + approach | 82 | 0.700 | 0.589 | 0.676 | +0.024 |
All three pre-registered predictions hold, and the shape is the finding. The probe is flat from the first interposed task onward — 0.681, 0.698, 0.700 — while the targeted baseline collapses monotonically — 0.950 → 0.676. The baseline was scoring an imminent continuation and loses most of its power once the answer is no longer imminent; the probe reads a disposition that persists. They cross between 56 and 82 tokens.
Model-dependent, and reported as such: both Qwen cells cross clearly (+0.084, +0.152), one Llama cell reaches −0.035, the other stays flat. Real on average and in two of four cells, not universal.
So the negative result is bounded, and the boundary is distance — not payload knowledge, which really did close. This reconciles the work with Kortukov et al., who probe multi-step-ahead behaviour and find internals useful: we find the same thing at depth. What we add is the crossover, and the observation that most probe evaluations sit below it.
what survivesThree results that held
Transfer has a measured price
Training a probe on one task family and deploying it on another costs 0.179 AUROC — 0.776 native against 0.595 across a domain change — while transfer within a domain across seeds costs nothing (0.780 vs 0.782). Validating a monitor on the distribution it was born in tells you little about any other. This was previously asserted; it is now measured.
Chain-of-thought protects, and we know why
Adoption falls 0.350 → 0.056 (Qwen) and 0.428 → 0.039 (Llama), p ≤ 1.9e−13. Three arms isolate the mechanism.
| arm | Qwen adoption | Qwen accuracy | p vs no-CoT | what it tests |
|---|---|---|---|---|
| no CoT | 0.350 | 0.183 | — | baseline |
| chain of thought | 0.056 | 0.828 | 4.9e−13 | — |
| silent reasoning | 0.328 | 0.167 | 0.388 | same computation, nothing written |
| padded | 0.311 | 0.172 | 0.311 | matched length, filler text |
| adjacent | 0.139 | 0.744 | 2.8e−06 | source restated beside the answer |
Length is not it — padding protects nothing. Abstract "reasoning" is not it either — asking for silent working protects nothing, and its accuracy stays at the no-CoT floor, so the model is not reasoning silently, it is not reasoning. The protection requires the tokens to be emitted. Restating the source beside the answer triples adoption but still protects, so salience-dilution is roughly a third of the effect.
Two safety behaviours, with planted ground truth
Prompt-injection compliance 0.750 (Qwen) and 0.583 (Llama); sycophantic capitulation after one round of user pushback 0.475 and 0.350.
scopeWhat this does not show
- The baseline needs the planted value. Realistic for prompt injection and document poisoning, where you can read what the document said. Unavailable when you do not know what payload to score.
- Every cell predicts a same-step outcome — the regime where the output distribution nearly is the answer, and the regime most probe papers evaluate in. Whether internals earn their place further ahead of the behaviour is the obvious next question. We tried to find that boundary and our attempt did not survive its own control.
- Two model families, 1.5B–14B. The ladder shows logP winning at every size, but nothing here reaches frontier scale.
provenanceWhere this sits
Kortukov et al. (2026), arXiv 2606.11172 show activation probes predicting future behaviour from intermediate reasoning steps at 64–91%, and steer on it. This does not contradict them: they predict multi-step-ahead behaviour, which is precisely the regime we identify as where internals should help. What we add is the comparison they do not run, and the finding that in the single-step regime the output distribution dominates.
Controls carried throughout: question-grouped cross-validation, group bootstrap, label-permutation nulls with their own confidence intervals, random-direction nulls at matched norm, margins combining both sources of variance, a nothing-loaded control condition, and a matched-norm random direction with coherence gating on every steered generation.