Interpretable Synthetic-Speech Detection with Concept Formation and Diffusion Prototypes

Abstract

We describe a synthetic-speech detector and an explanation layer that makes its decisions open to inspection. The detector averages three fine-tuned self-supervised speech models. We trained it on data from which we removed format shortcuts and to which we added 11,900 copy-synthesis fakes. It scores a minimum detection cost (minDCF) of 0.0317 on the organizers' test set and 0.0274 [0.0231, 0.0306] on 27,779 held-out In-the-Wild recordings.

The explanation layer organizes the detector's representation into a COBWEB concept hierarchy and explains each clip with diffusion prototypes named by those concepts. It agrees with the detector's decisions at κ = 0.985. We study all 5,013 prototypes that explain the 1,671 test clips. They fall into eight families, five synthetic and three real, each tied to a class of synthesis system or of real speech. Every synthetic family matches a measurable acoustic profile. Relative to real speech, its clips show higher estimated SNR, lower long-term spectral ripple and more variable noise floors. Family-specific markers sit on top of that profile, such as a smooth LPC residual for diffusion TTS. Twenty-one case studies show how prototypes separate synthesis systems, keep degraded real speech real, and single out clips the detector may have missed.

Contents

1. Title and abstract p. 1
2. 1 Introduction p. 2
3. 2 Background p. 3
4. 3 Detection system p. 4
5. 4 The concept space p. 5
6. 5 Prototype inspection p. 6
7. 6 Prototype families p. 7
8. 7.1 Case studies: commercial cloners p. 8
9. 7.2 Case studies: open cloning and conversion p. 9
10. 7.3 Case studies: diffusion TTS p. 10
11. 7.4 Case studies: real speech and channels p. 11
12. 7.5 Case studies: review and failure p. 12
13. 8 Evaluation of the explanations p. 13
14. 9 Submitted system and ablations p. 14
15. 10 Discussion and limitations p. 15
16. 11 Conclusion and future work p. 16
17. References and appendix p. 17

1Introduction

Zero-shot voice cloning now requires only seconds of reference audio. An analyst who receives a suspicious recording needs two answers: how likely the recording is to be synthetic, and on what evidence. The challenge organizers made the second answer explicit. They asked teams to state what worked, to explain why, and to minimize black-box detectors.

Detectors built on large self-supervised speech models generalize better than earlier hand-crafted systems [Tak et al. 2022a; Ge et al. 2025]. They remain opaque, and they readily learn properties of a dataset instead of properties of synthesis [Müller et al. 2021]. Our detector follows this line and adds three safeguards: shortcut removal, controlled synthetic data and a calibrated ensemble. The larger part of this report concerns inspection. We ask what the detector's representation encodes and whether a person can read it.

We answer with two tools from the study of categorization. COBWEB [Fisher 1987] builds a concept hierarchy incrementally, as a model of how people form categories. Diffusion prototypes, obtained by test-time concept discovery (TTCG) [Wang et al. 2026b], describe an individual clip as a composition of a few modes of the learned distribution. We name each prototype by the COBWEB concept that contains it, so every explanation refers to training audio a person can listen to.

1.1 Contributions

1. A shortcut-free detector. An audit shows that file timestamps alone separate the provided classes. A canonical view removes such cues, and copy-synthesis fakes teach the synthesis step itself. The calibrated ensemble reaches 0.0274 minDCF on 27,779 held-out recordings (§3).
2. A concept space with a held-out basic level. We fit a COBWEB hierarchy on the detector's own representation. We select each clip's basic level by pointwise mutual information on held-out data, which peaks at an interior depth where the closed-form criterion does not (§4).
3. Prototype inspection. We implement TTCG for this space and name its prototypes by concepts (§5).
4. Prototype families and their acoustic criteria. We group all 5,013 test-set prototypes into eight families and measure what acoustically distinguishes the audio each family explains (§6). Twenty-one case studies examine individual decisions (§7).
5. Evaluated explanations. We measure fidelity, faithfulness, stability, label-free structure and three cognitive capabilities against baselines (§8).

2Background

2.1 Task and metric

The system receives an audio file of at least two seconds of English speech and outputs P(synthetic). The organizers rank systems with the ASVspoof 5 detection cost function at a spoof prior π = 0.3, C_miss = 1 and C_fa = 4 [Wang et al. 2024]. The package treats bona fide speech as the target class, so its false-alarm cost applies to a synthetic clip that passes as real:

where FPR is the share of real clips flagged and FNR the share of synthetic clips passed. minDCF is the minimum over τ. After calibration at the evaluation prior, the Bayes-optimal rule flags a clip when its expected cost of acceptance exceeds that of rejection:

2.2 Prototypes, exemplars and the basic level

People classify new instances by their resemblance to a category's central tendency, its prototype. Members close to the prototype are judged more typical and are learned faster [Posner and Keele 1968; Rosch and Mervis 1975]. Exemplar theories store the instances themselves and classify by similarity to them [Medin and Schaffer 1978; Nosofsky 1986]. Categories also form hierarchies, and people favour one level of such a hierarchy, the basic level, at which categories are most informative [Rosch et al. 1976]. Prototype-based networks carry these ideas into machine learning and explain a decision by pointing at learned prototypes [Snell et al. 2017; Chen et al. 2019].

2.3 COBWEB

COBWEB [Fisher 1987] sorts each new instance down a concept hierarchy. At every node it chooses among four operators: add the instance to the best child, create a new child, merge the two best children, or split the best child. It keeps the operator that maximizes category utility [Gluck and Corter 1985], the gain in predictability of attribute values that a partition affords:

For continuous attributes, each concept stores a diagonal Gaussian, and utility rewards children that are tighter than their parent [Gennari et al. 1989]. We use the continuous implementation of the Teachable AI Lab (cobweb-private). When labels are supplied, its objective adds a label-entropy term; §4.5 removes that term to test its influence. We measure how informative a concept c is for an instance x by pointwise mutual information with the root, and define the basic level of x as the node on its path where this quantity, estimated on held-out data, peaks [Wang et al. 2026a]:

2.4 Diffusion models as concept formation

A denoising diffusion model [Ho et al. 2020] learns to reverse a gradual noising process. With \(\bar\alpha_t\) the cumulative signal level at step t,

where the second expression is Tweedie's posterior mean [Efron 2011]. Wang, Singaravadivelan and MacLellan [Wang et al. 2026a] argue that the noised marginals of such a model describe a hierarchy of Gaussian prototypes, with high noise levels corresponding to general concepts and low noise levels to specific ones. Test-time concept discovery and composition (TTCG) [Wang et al. 2026b] uses this view to explain a single input: it climbs the score function from a noised copy of the input to nearby modes, selects a few of them and composes them. §5 describes our implementation, and §5.4 tests the noise-level hypothesis in our space.

3Detection system

3.1 Data

The organizers supplied DiffSSD [Bhagtani et al. 2024], about 70,000 synthetic clips from ten systems, and 242 real clips resampled from LJSpeech. We added 19,575 real recordings: LJSpeech (13,100), the ten LibriSpeech speakers that DiffSSD clones (1,152) and 80 further LibriSpeech speakers (5,323). We generated 11,900 copy-synthesis fakes by re-synthesizing training-split real recordings with two HiFi-GAN models, DiffWave and Vocos [Wang and Yamagishi 2023]. Each such fake shares its speaker and content with a real recording. The In-the-Wild dataset [Müller et al. 2022], 31,779 web recordings of real and synthetic speech by public figures, serves only for evaluation.

An audit preceded training. Byte hashes, decoded-audio hashes and acoustic landmark matching against 89,817 reference recordings found no test audio in any training corpus. The audit also found strong shortcuts in DiffSSD: file-system timestamps alone separate the classes (AUC 1.0), and sample rate, codec, level, leading silence and duration differ by class.

3.2 Canonical view and augmentation

Every clip passes through one canonical view: decoding to 16 kHz mono, trimming of edge silence, a crop to a test-like duration during training, a 7 kHz low-pass filter, DC removal, peak normalization and one-LSB dither. During training, 60% of clips also pass through one or two channel operations: lossy codecs, telephony, band-limiting, resampling, spectral tilt, noise, mains hum, reverberation or clipping. Both classes receive the same operations, so a degraded real recording remains a real example.

3.3 Detectors, fusion and calibration

We fine-tune three AntiDeepfake models [Ge et al. 2025] at a learning rate of 2 × 10⁻⁶ for three epochs, with fixed sampling shares per data family. Each produces a synthetic logit ℓ_k. We z-normalize the logits, average them and apply Platt scaling at the evaluation prior [Platt 1999]:

A router re-scores uncertain clips (0.05 < P < 0.8) and clips longer than six seconds in two-second windows, and cross-checks lossy or band-limited files. It records every decision in a per-file trace.

Plain fine-tuning degraded out-of-domain performance at every epoch (XLS-R-1B: 0.050 to 0.129). With copy-synthesis fakes it held or improved (0.050 to 0.039). The ensemble beats its best member on paired tests, by 0.012 [0.001, 0.020] on clean audio and 0.022 [0.004, 0.037] under perturbation.

4The concept space

4.1 Representation

The explanation layer reads the time-averaged final-layer representation of the XLS-R-2B detector, the input of its linear classification head. We standardize it and whiten it with principal component analysis to 32 dimensions, fitted on training clips. A linear read-out from these 32 dimensions reproduces the detector's logit with R² = 0.99997, so the space holds essentially all the information behind the decision.

4.2 Hierarchy

We fit a COBWEB hierarchy on 11,999 training clips, with source and channel as labels, and hold out 999 clips to estimate the basic level. Test clips enter the hierarchy by categorization alone: they are sorted down the tree without updating it.

4.3 The basic level

Figure 2 contrasts two criteria for the basic level. The expected pointwise mutual information computed in closed form under each concept's own Gaussian rises with depth at every level, because a narrower Gaussian always appears more informative about its own members. Under that criterion the basic level is a single-clip leaf. Estimated on held-out clips, pointwise mutual information rises to a peak at depth 7 and then falls. The median concept at that depth holds eight training clips. We use the held-out criterion throughout.

4.4 The concept atlas

Three properties of the atlas bear on interpretation. First, the root divides first by authenticity and by corpus: real LJSpeech (0.1), synthetic speech in the LJSpeech voice together with the commercial cloners (0.2), open cloners and voice conversion (0.3), and real LibriSpeech (0.4). Second, the commercial cloners ElevenLabs and PlayHT imitate LibriSpeech speakers, yet they sit under concept 0.2 beside diffusion TTS in the LJSpeech voice. The hierarchy therefore groups them by how they render audio rather than by whose voice they imitate. Third, channel conditions form their own sub-concepts on both sides of the authenticity split: reverberant real speech (0.1.2), noisy real speech (0.1.3, 0.4.3) and reverberant clones (0.3.3).

4.5 Structure without labels

Because the objective includes a label term, we refit the hierarchy on the same clips with no labels at all and named its concepts afterwards by their members. Table 3 reports purity by depth.

Without labels, the first split already separates real from synthetic speech at 98.9% purity, and deeper levels separate systems and speaker populations. The representation comes from a supervised detector. The result therefore shows that this representation is organized by authenticity; it does not show that authenticity emerges from raw audio.

5Prototype inspection

Our explanation of a clip x has three parts, which we compute as follows.

5.1 Discovery

We train a denoising diffusion model on the 32-dimensional concept space. For each clip we noise its representation to levels t ∈ {50, 75, …, 400} and climb the learned score function to a nearby mode x^*. Tweedie's formula maps each mode to a prototype mean m = x^*/√ᾱ_t. The diagonal of the denoiser's Jacobian, estimated with Hutchinson probes [Hutchinson 1989], gives its covariance.

5.2 Selection and composition

We select up to three prototypes by greedy maximization of a facility-location objective over the 32 dimensions r. The standard normal root distribution provides the baseline b_r that a prototype must improve on. The objective is monotone and submodular, so greedy selection comes within a factor 1 − 1/e of the optimum [Nemhauser et al. 1978]:

Each dimension is then assigned to the prototypes by a softmax over their log-likelihoods, and a prototype's share is the fraction of dimensions it accounts for. The composed explanation is a product of experts [Hinton 2002].

5.3 Naming and scoring

Each prototype mean is categorized into the COBWEB hierarchy. The concept it reaches names the prototype and supplies its sources, channels, synthetic share p_fake(c) and most typical training clips. The explanation's own synthetic probability weights the concepts by share:

This quantity is reported beside the detector's probability in every case study. Agreement between the two supports the decision, and a large gap marks the clip for review (§7.5).

5.4 Noise level and generality

The hypothesis of §2.4 predicts that prototypes found at high noise levels are named by larger, shallower concepts. Table 4 tests it on the 5,013 test-set prototypes.

The prediction fails in this space. Concepts named by high-noise prototypes are, if anything, smaller and deeper, and they are more often synthetic. Most prototypes (59%) are found at the lowest noise level. We therefore read the noise level as a search radius rather than as a level of generality: larger radii reach modes of other, often synthetic, concepts. §7.4 shows the consequence for real clips.

6Prototype families and their criteria

6.1 Assignment

We assign each prototype to a family by the dominant source of the concept that names it. Thirteen sources yield eight families: two commercial cloners, open zero-shot cloners (XTTS v2, YourTTS), unit-based cloning and conversion (UnitSpeech, OpenVoice v2), diffusion TTS in the LJSpeech voice (DiffGAN-TTS, Grad-TTS, ProDiff, WaveGrad 2), and three real-speech families. A clip's dominant family is the family with the largest summed prototype share. Its purity is that share.

The families align with the detector's decisions. It flags 96.7–100% of the clips that synthetic families dominate and 0–0.4% of those the real LibriSpeech families dominate. The real LJSpeech family is the exception: it dominates only ten clips, 40% of them flagged, and its concepts are 48% reverberant. §7.5 examines these clips. Among the synthetic families, prototypes of the diffusion TTS family are found at the highest noise levels (31% at t ≥ 150) and most often name channel concepts (41%, mostly codec).

6.2 Acoustic criteria

To state what each family captures in measurable terms, we compare the test clips a synthetic family dominates (purity ≥ 0.66) with the 978 clips the real LibriSpeech families dominate. We use 20 interpretable features from the signal-processing track: prosody, background noise, spectral shape, the LPC residual and phase. The effect size is Cliff's δ, the probability that a family clip exceeds a real-attributed clip minus the reverse, with a 300-sample bootstrap interval. No test labels exist, so the comparison describes the audio each family explains rather than ground truth.

6.3 Findings

A shared synthetic profile. All five synthetic families differ from real-attributed clips in the same direction on five features: higher estimated SNR (δ = +0.48 to +0.93), lower long-term spectral ripple, more variable noise floors, flatter high bands and a narrower spread of spectral flux. All 25 of these family-by-feature differences have intervals that exclude zero. The prototypes therefore encode a common property of the synthesized audio in this data: it is cleaner and spectrally smoother than the recordings it imitates, and its noise floor varies more.

Family-specific criteria. Diffusion TTS in the LJSpeech voice is the most extreme case. Its clips are almost noiseless (median SNR 38.3 dB, δ = +0.93), have a smooth LPC residual (kurtosis 2.2 against 4.5, δ = −0.78), carry less energy below 250 Hz and contain more pauses. PlayHT also shows a smooth LPC residual (δ = −0.48), a mark of vocoder excitation. The open cloners and the conversion systems show flatter background noise (δ = +0.31 and +0.42). The real LJSpeech family differs little from real LibriSpeech: only frame-loudness variability and LPC residual kurtosis differ clearly, on ten clips.

A caution. A cleaner signal is not in itself evidence of synthesis. Studio recordings are also clean, and a detector that relied on cleanliness alone would flag them. The detector's out-of-domain accuracy (§3) and the channel case studies (§7.4) indicate that it relies on more than cleanliness. The stress view in §9, however, shows that simulated room acoustics still degrade it.

7.1Commercial cloners

ElevenLabs and PlayHT imitate LibriSpeech speakers from short references. In the atlas they share concept 0.2.2. Their prototypes are found at low noise levels, name small concepts (median six clips) and are 82% clean.

ElevenLabs: criteria

The family dominates 122 test clips (median detector P = 0.997, 96.7% flagged). Its clearest acoustic criteria against real-attributed clips:

Medians over the 52 test clips this family dominates (purity ≥ 0.66) and the 978 clips the real LibriSpeech families dominate.

Case 1. A pure ElevenLabs explanation

All three prototypes are ElevenLabs concepts found at t = 50, and so is the basic-level concept (23 clips). The clip meets the family's criteria: high estimated SNR and low spectral ripple. Detector and prototypes agree at P ≈ 1.

Case 2. ElevenLabs within the commercial-cloner concept

The basic-level concept holds 115 clips, 90% ElevenLabs and 8% PlayHT. The prototypes refine it to three ElevenLabs concepts of 3 to 22 clips. The explanation therefore moves from the commercial cloners in general to one system.

PlayHT: criteria

The family dominates 87 test clips (median detector P = 0.998, 98.9% flagged). Its clearest acoustic criteria against real-attributed clips:

Medians over the 39 test clips this family dominates (purity ≥ 0.66) and the 978 clips the real LibriSpeech families dominate.

Case 3. PlayHT under a codec

The largest prototype (48%) is a five-clip PlayHT concept found at t = 75 whose clips are 40% codec-processed. PlayHT's clearest criterion is its smooth LPC residual (§6.3), and the attribution holds for a concept that is partly codec-processed.

Case 4. PlayHT and UnitSpeech share deep concepts

The basic-level concept is a two-clip concept, half PlayHT and half UnitSpeech. One prototype names a 57-clip PlayHT and UnitSpeech concept. Commercial and open unit-based cloners meet at deep levels of the hierarchy. The explanation still assigns the clip to PlayHT through its largest prototype, a single PlayHT clip.

7.2Open zero-shot cloning and voice conversion

XTTS v2 and YourTTS are open zero-shot cloners. UnitSpeech clones through discrete speech units, and OpenVoice v2 converts a base voice's tone colour. In the atlas they occupy branch 0.3, apart from the commercial cloners. Their prototypes are more often channel concepts (24–26%, mostly codec) than those of the commercial cloners.

XTTS v2 / YourTTS: criteria

The family dominates 113 test clips (median detector P = 0.999, 100.0% flagged). Its clearest acoustic criteria against real-attributed clips:

Medians over the 65 test clips this family dominates (purity ≥ 0.66) and the 978 clips the real LibriSpeech families dominate.

Case 5. YourTTS through a codec

The basic-level concept is a single YourTTS clip. Two prototypes (66%) name a six-clip YourTTS and XTTS v2 concept that is half codec-processed. A third prototype, found at t = 200, names a seven-clip YourTTS concept. The family's criteria include flatter background noise (§6.3).

Case 6. XTTS v2

All three prototypes name XTTS v2 concepts. The largest (46%) also contains UnitSpeech clips. The explanation isolates the cloning system even though XTTS v2 and YourTTS imitate the same ten speakers.

UnitSpeech / OpenVoice v2: criteria

The family dominates 108 test clips (median detector P = 0.999, 100.0% flagged). Its clearest acoustic criteria against real-attributed clips:

Medians over the 58 test clips this family dominates (purity ≥ 0.66) and the 978 clips the real LibriSpeech families dominate.

Case 7. OpenVoice v2 conversion explained by unit-based texture

The basic-level concept is OpenVoice v2 (73%) with YourTTS (24%). The largest prototype (44%), however, names a 102-clip concept of UnitSpeech and PlayHT. Voice conversion and unit-based cloning thus share a region of the space. The family's criteria describe that shared texture: high high-band flatness and a narrow spread of spectral flux.

Case 8. UnitSpeech

Two prototypes name the clip's own basic-level concept, 45 clips of UnitSpeech (69%) and XTTS v2. A third names a 21-clip UnitSpeech and PlayHT concept. All shares point to unit-based cloning.

7.3Diffusion TTS in the LJSpeech voice

DiffGAN-TTS, Grad-TTS, ProDiff and WaveGrad 2 all speak in the LJSpeech voice. They form the atlas concepts 0.2.1 and 0.2.3. On the test set this family dominates only 41 clips, but its acoustic criteria are the strongest of any family.

LJ-voice diffusion TTS: criteria

The family dominates 41 test clips (median detector P = 0.997, 97.6% flagged). Its clearest acoustic criteria against real-attributed clips:

Medians over the 23 test clips this family dominates (purity ≥ 0.66) and the 978 clips the real LibriSpeech families dominate.

Case 9. Grad-TTS through a codec

All three prototypes name one two-clip Grad-TTS concept, half codec-processed, found at t = 50 and 75. The explanation is as specific as the hierarchy allows. The clip fits the family's criteria: very high estimated SNR and a smooth LPC residual.

Case 10. A composition of system and channel

The three prototypes name three different concepts: Grad-TTS under a codec (36%), ProDiff and PlayHT under reverberation (39%), and WaveGrad 2 with Grad-TTS in clean audio (25%). The composition separates the synthesis system from the channel. Each dimension goes to the concept that explains it best.

Case 11. Diffusion-vocoder texture inside a commercial clone

The basic-level concept is commercial (ElevenLabs and PlayHT, 50% each). All three prototypes, however, name WaveGrad 2 and Grad-TTS concepts, two of them under a codec. The prototypes thus identify a vocoder-level texture that the commercial clip shares with diffusion TTS. The basic-level concept alone would not reveal it.

7.4Real speech, channels and search radius

A detector for this task must keep degraded real speech real. The cases below examine real clips whose prototypes name channel concepts, a synthetic clip under a channel, and real clips that acquire a synthetic prototype at a large search radius. Across the test set, 155 prototypes name real concepts dominated by a channel condition, and 225 name synthetic concepts of that kind.

Case 12. Real recordings of a cloned speaker

All prototypes and the basic-level concept consist of real LibriSpeech recordings of the speakers the cloners imitate. This family dominates 682 test clips, more than any other, and the detector flags none of them. A familiar voice therefore does not by itself pull a clip toward the synthetic concepts.

Case 13. Real speech through a telephone channel

The largest prototype (39%) names a five-clip concept of real speech that is 60% telephony-processed. The explanation recognizes the channel and keeps the authenticity: every concept is real.

Case 14. Real speech with mains hum

The largest prototype (49%) names a real concept whose clips all carry mains hum. Hum was part of the training augmentation for both classes, and the hierarchy formed a real-speech concept for it.

Case 15. Real speech after resampling

Two prototypes (76%) name a single real clip that was resampled. Resampling removes high-frequency content, one of the cues in the shared synthetic profile. The clip nevertheless stays among real concepts, and the detector scores it 0.0003.

Case 16. Synthetic speech under spectral tilt

The largest prototype names an ElevenLabs concept whose clips are two-thirds tilt-processed. The channel changes the spectral balance but not the attribution. A third prototype found at t = 250 names a UnitSpeech concept. At large radii the search reaches other synthetic modes (§5.4).

Case 17. A synthetic prototype at a large search radius

Two prototypes name real concepts. The third, found at t = 375, names a ten-clip ProDiff concept and receives 40% of the dimensions, so the prototype probability rises to 0.40 while the detector gives 0.0002. Among the 25 test clips whose prototype probability lies between 0.2 and 0.8, 14 follow this pattern: their synthetic prototype is found at t ≥ 150, at a larger radius than every real prototype of the clip. Such prototypes limit the prototype probability as a stand-alone score and support reading the noise level as a search radius.

7.5Uncertainty, disagreement and failure

The explanation's value is greatest where the detector is least certain. Seventeen test clips have detector probabilities between 0.05 and 0.8. In eight clips the prototype probability and the detector differ by more than 0.5. No test labels exist, so these cases identify clips for review; they cannot confirm errors.

Case 18. Reverberant real LJSpeech near the threshold

All three prototypes name real LJSpeech concepts; two of them (63%) name a 40-clip concept that is 60% reverberant. The detector gives 0.44, above the 0.2 threshold. Reverberant real speech is the detector's weakest condition (In-the-Wild minDCF 0.25), and the explanation names that condition directly.

Case 19. A possible false alarm on reverberant speech

The detector flags this clip at 0.97. Two prototypes (66%) name a 37-clip real concept that is half reverberant. Only a single-clip Grad-TTS concept under reverberation, found at t = 250, is synthetic. The prototype probability is 0.40. The explanation attributes the score to the room rather than to synthesis.

Case 20. Reverberant real LibriSpeech

All prototypes are real concepts under reverberation, and the prototype probability is zero. The detector gives 0.56. The case repeats the pattern of case 19 for another speaker population.

Case 21. The explanation that exposed a scoring defect

Our first scorer cropped each batch to its shortest clip, so a clip's score depended on its neighbours. It assigned this clip 0.009, while every prototype named a synthetic concept of voice clones under a codec. The contradiction led us to the defect. The corrected scorer assigns 0.996; the correction changed 3 of 1,671 test decisions.

Candidates for missed commercial clones

In 6 clips the prototypes point firmly to a synthetic family while the detector passes or nearly passes the clip. All belong to the commercial-cloner families, the detector's hardest generators in development (ElevenLabs holdout minDCF 0.104).

Four of the six carry channel concepts (noise, resampling, tilt) among their prototypes. A review queue ordered by this disagreement is the error-triage mechanism evaluated in §8.4.

8Evaluation of the explanations

8.1 Fidelity

8.2 Faithfulness and stability

We test faithfulness by deletion [Samek et al. 2017] on the 66 of 3,000 explanations that mix synthetic and real concepts. Deleting the dimensions the explanation assigns to synthetic concepts lowers the detector's logit more than deleting those assigned to real concepts: by 0.79 SD [0.67, 0.89], in 92% of cases. Random dimensions give effects of 0.20 [0.14, 0.24] and 0.18 [0.13, 0.22] SD. Across three insertion orders the hierarchy changes (partition ARI 0.03–0.04), but the leading source of an explanation agrees for 94–95% of clips.

8.3 Failure localization

On 2,794 held-out and In-the-Wild clips with 73 errors, five concepts contain 27% of the errors. The largest are a 1,177-clip concept of XTTS v2 and YourTTS clones (14% error) and two small concepts of reverberant real speech (46% and 50% error). The concepts name the detector's weak conditions directly.

8.4 Cognitive capabilities

With a single example, COBWEB attributes 7% of the withheld generator's clips, against 0% for kNN and 1% for logistic regression. The two tie at three examples, and logistic regression leads from ten. Attribution of known generators stays at 0.87, so insertion causes no forgetting. For novelty, negative basic-level mutual information separates clips of an unseen generator with AUROC 0.87 ± 0.06, against 0.49 for a Mahalanobis distance. For error triage on In-the-Wild, ranking clips by disagreement between explanation and detector reaches an average precision of 0.38, nine times that of a random ranking (0.04). Combined with the detector's own uncertainty, it recovers 95% of errors in the top 10% of clips.

9Submitted system and ablations

System A, the three-model ensemble of §3, is the submitted system. Two ablations change only the ensemble members. Ablation B replaces the XLS-R-1B member with one trained with RawBoost augmentation [Tak et al. 2022b], chosen on validation data under criteria fixed in advance. Ablation C adds a second-seed XLS-R-1B and a WiSE-FT XLS-R-2B [Wortsman et al. 2022].

The RawBoost member alone improved perturbed In-the-Wild minDCF from 0.140 to 0.121. The paired 95% interval, [−0.033, 0.004], includes zero, so it failed its predeclared criterion; its backbone had already received RawBoost during post-training. The larger ensemble trades a slight clean-audio loss for a slight gain under perturbation. Neither ablation improves on System A beyond sampling noise.

A stress view built from channels that no model trains on (simulated rooms, unseen codecs, packet loss, gain control) degrades every system to minDCF 0.26–0.31, and simulated rooms alone to 0.51–0.59. This agrees with the reverberation cases of §7.5.

10Discussion and limitations

What the prototypes reveal. The prototypes attribute test clips to recognizable classes of synthesis. The attribution matches measurable acoustic differences, and it separates the synthesis system from the channel it passed through (cases 10, 13–16). The hierarchy groups systems by how they render audio rather than by whose voice they imitate (§4.4, case 11). This is what distinguishes the layer from a post-hoc saliency map: each explanation refers to concepts formed from, and exemplified by, real training audio.

What the prototypes cannot establish. No test labels exist. The acoustic criteria describe the audio each family explains, and the disagreement cases (Table 6) are candidates for review, not confirmed errors. The shared synthetic profile includes cleanliness, which also characterizes studio recordings. The detector's out-of-domain accuracy argues against reliance on cleanliness alone, but the stress results show that room acoustics remain a weakness.

Method limitations. The shipped hierarchy is fitted with labels; the label-free analysis bounds but does not remove this dependence. The noise-level hypothesis of diffusion concept formation does not hold in our space (Table 4), and prototypes found at large search radii can name synthetic concepts for real clips (case 17). The explanation layer reads one detector (XLS-R-2B) of the three. The LJ-voice diffusion family dominates only 41 test clips and the real LJSpeech family ten, so their criteria rest on small samples.

11Conclusion and future work

A calibrated ensemble of fine-tuned speech models, trained on shortcut-free data with our own synthetic examples, reaches minDCF 0.0317 on the organizers' test set and 0.0274 on 27,779 held-out recordings. A concept-formation layer makes its decisions inspectable. It agrees with the detector at κ = 0.985 and explains each clip through prototypes named by learned concepts. Across the test set, the prototypes form eight families with measurable acoustic criteria, and 21 case studies show how they separate synthesis systems from channels and single out clips for review.

• Room acoustics. Training with measured room impulse responses addresses the clearest weakness, found by both the stress view and the explanation layer.
• Radius-aware prototypes. Restricting or down-weighting high-noise prototypes would remove the spurious synthetic attributions of case 17.
• Explaining the full ensemble. Concept spaces for the XLS-R-1B and MMS-1B members would let explanations cover every component of the score.
• Newer generators. Open-weight systems released in 2025–26 would test whether the synthetic profile of §6.3 holds.

References

• Bhagtani et al. 2024. Bhagtani et al. DiffSSD: a diffusion-based dataset for speech forensics. 2024.
• Chen et al. 2019. Chen et al. This looks like that: deep learning for interpretable image recognition. NeurIPS, 2019.
• Efron 2011. Efron. Tweedie's formula and selection bias. JASA, 2011.
• Fisher 1987. Fisher. Knowledge acquisition via incremental conceptual clustering. Machine Learning, 1987.
• Ge et al. 2025. Ge, Wang, Liu and Yamagishi. Post-training for deepfake speech detection (AntiDeepfake). 2025.
• Gennari et al. 1989. Gennari, Langley and Fisher. Models of incremental concept formation. Artificial Intelligence, 1989.
• Gluck and Corter 1985. Gluck and Corter. Information, uncertainty, and the utility of categories. Proc. Cognitive Science Society, 1985.
• Hinton 2002. Hinton. Training products of experts by minimizing contrastive divergence. Neural Computation, 2002.
• Ho et al. 2020. Ho, Jain and Abbeel. Denoising diffusion probabilistic models. NeurIPS, 2020.
• Hutchinson 1989. Hutchinson. A stochastic estimator of the trace of the influence matrix. 1989.
• Kong et al. 2020. Kong, Kim and Bae. HiFi-GAN. NeurIPS, 2020.
• Kong et al. 2021. Kong et al. DiffWave. ICLR, 2021.
• Medin and Schaffer 1978. Medin and Schaffer. Context theory of classification learning. Psychological Review, 1978.
• Müller et al. 2021. Müller et al. Speech is silver, silence is golden: what do ASVspoof-trained models really learn? 2021.
• Müller et al. 2022. Müller et al. Does audio deepfake detection generalize? Interspeech, 2022.
• Nemhauser et al. 1978. Nemhauser, Wolsey and Fisher. An analysis of approximations for maximizing submodular set functions. Math. Programming, 1978.
• Nosofsky 1986. Nosofsky. Attention, similarity, and the identification–categorization relationship. JEP: General, 1986.
• Platt 1999. Platt. Probabilistic outputs for support vector machines. 1999.
• Posner and Keele 1968. Posner and Keele. On the genesis of abstract ideas. J. Experimental Psychology, 1968.
• Rosch and Mervis 1975. Rosch and Mervis. Family resemblances. Cognitive Psychology, 1975.
• Rosch et al. 1976. Rosch et al. Basic objects in natural categories. Cognitive Psychology, 1976.
• Samek et al. 2017. Samek et al. Evaluating the visualization of what a deep neural network has learned. IEEE TNNLS, 2017.
• Siuzdak 2023. Siuzdak. Vocos. 2023.
• Snell et al. 2017. Snell, Swersky and Zemel. Prototypical networks for few-shot learning. NeurIPS, 2017.
• Tak et al. 2022a. Tak et al. Automatic speaker verification spoofing and deepfake detection using wav2vec 2.0 and data augmentation. Odyssey, 2022.
• Tak et al. 2022b. Tak et al. RawBoost: a raw data boosting and augmentation method. ICASSP, 2022.
• Wang and Yamagishi 2023. Wang and Yamagishi. Spoofed training data for speech spoofing countermeasure can be efficiently created using neural vocoders. ICASSP, 2023.
• Wang et al. 2024. Wang et al. ASVspoof 5: crowdsourced speech data, deepfakes, and adversarial attacks at scale. 2024.
• Wang et al. 2026a. Wang, Singaravadivelan and MacLellan. Diffusion models and concept formation (DMCF). 2026.
• Wang et al. 2026b. Wang, Gupta, Zhu and MacLellan. Test-time compositional generalization in diffusion models via concept discovery (TTCG). 2026.
• Wortsman et al. 2022. Wortsman et al. Robust fine-tuning of zero-shot models (WiSE-FT). CVPR, 2022.

The full bibliography is in docs/APPROACH.md.

Appendix: reproducibility

• The Docker image scores a directory of audio without network access and matches the reference run on all 1,671 test files to within 3.2 × 10⁻⁶.
• Every results directory records its code version, input hashes, package versions and command (provenance.json).
• Re-training the XLS-R-1B detector from the repository reproduced the stored validation results to four decimal places at every epoch.
• Explanations: results/concepts/test_explanations.jsonl. Families, criteria and cases: results/concepts/case_studies/, built by scripts/prototype_case_studies.py. Evaluation of explanations: results/concepts/experiments/.
• Submission files: submission/candidates/SideQuests_predictions_final_{A,B,C}.tsv.