Hand-measured · the study's ground truth (will be updated with patient language)
Phrase dataset
Every hand-built sentence pair.
Each pair was built by hand and measured on gemma-2-2b via the Neuronpedia circuit tracer; the table records the observed next token, its probability, and, where captured, a link to the live trace. The simulated scenarios extend this set by machine.
The starkest case in the set illustrates a change in the recommended clinical advice. “Since her urinary tract was completely blocked up, they had to urgently call a” continues with uro(logist) at 20%; phrase it as “her water was completely blocked up” and the model calls a plumber — at 68%. The patient language changed what was recommended. (Pair 16 below.)
All pairs
measured on: gemma-2-2b · gemma scope transcoders → observed next token · p = observed next-token probability · via the neuronpedia circuit tracer
| Pair | Patient phrasing | Clinical phrasing |
|---|
Next-token probabilities as observed at measurement time.
prompts shown verbatim, including intentional misspellings