Simulated series · generated stress pairs · live renders · simulated data
Simulated scenarios
Stress scenarios authored by an LLM, validated, traced live on gemma-2-2b, and kept apart from the hand-measured set.
View generation methodology
This section adds simulated scenarios to the hand-measured dataset. Claude writes new patient-vs-clinical pairs in the same format — one term swapped inside an identical frame, ending at the next-token probe. Each candidate must pass automatic checks (single swap, correct ending, term-verbatim, no duplicates of the measured set) before it’s accepted and traced live on gemma-2-2b.
Each scenario shows the two phrasings, the traced probabilities with the wording gap, and the generator’s reason the swap should matter.
Wording gap: the percentage points of next-word probability the clinical answer loses when the sentence is worded the way an everyday patient would say it, as opposed to standard medical terms.
In the table, the model hedges (top answer holds, loses probability) or redirects; the urgency column marks a redirect down the care ladder as a downgrade.
traced model: gemma-2-2b · gemma scope transcoders → prob = target-token probability read from the live trace · wording gap = patient − clinical
Key example
When the advice itself changes
Some redirects go further — the top answer lands on a different object altogether. Tier labels: clinician review pending.
| Swap (patient → clinical) | Target |
|---|
click a Sim number to jump to its full trace · glyph: ● clinical → ○ patient target probability on a shared 0–1 axis (dashed = patient below the traced spread) · * intended target fell below the traced spread; measurement anchored on the clinical top prediction · screened-out rows (failed the measurement screen; clinical trace kept) are hidden by default — the “hide screened-out” toggle above shows them
The key example, full size
Repeatability —
SIMULATED DATA · these phrasings were written by an LLM (generation provenance: the batch records in data/provenance.json) and passed the generator's validators; they are not statements from patients.