Each item gives a model one chapter of a public-domain novel and a deliberately divergent brief for the next; the continuation cannot be retrieved from training data. One LLM judge grades each chapter on compliance with the brief and on eight dimensions of craft, always beside the author’s real next chapter, and the headline score is their product. The full recipe is on the Method page.
Models ranked by headline score at their best reasoning-effort setting: compliance × craft on a 0–100 scale, one judge, ten items.
| Rank | Model | Headline | Compliance | Craft | Cost∗ |
|---|---|---|---|---|---|
| Claude Opus 5 xhigh Anthropic · xhigh effortcompliance 98 · craft 80 · 2,450 wordsmedium71.5high75.5 | 79.2±1.5 | 98 | 80 | $3.07 | |
| · | Claude Opus 5 medium Anthropic · medium effortcompliance 96 · craft 75 · 2,433 words | 71.5±2.1 | 96 | 75 | $1.34 |
| · | Claude Opus 5 high Anthropic · high effortcompliance 98 · craft 77 · 2,430 words | 75.5±1.4 | 98 | 77 | $2.02 |
| Claude Sonnet 5.5 xhighNew Anthropic · xhigh effortcompliance 99 · craft 79 · 2,255 wordsmedium68.9high74.1 | 78.3±1.5 | 99 | 79 | $3.15 | |
| · | Claude Sonnet 5.5 medium Anthropic · medium effortcompliance 96 · craft 72 · 1,820 words | 68.9±2.6 | 96 | 72 | $0.32 |
| · | Claude Sonnet 5.5 high Anthropic · high effortcompliance 97 · craft 76 · 2,592 words | 74.1±2.9 | 97 | 76 | $0.84 |
| Claude Opus 5.5 xhigh Anthropic · xhigh effortcompliance 100 · craft 77 · 2,293 wordsmedium75.2high74.9 | 77.4±1.7 | 100 | 77 | $5.12 | |
| · | Claude Opus 5.5 medium Anthropic · medium effortcompliance 98 · craft 76 · 2,536 words | 75.2±0.9 | 98 | 76 | $1.22 |
| · | Claude Opus 5.5 high Anthropic · high effortcompliance 100 · craft 75 · 2,417 words | 74.9±1.8 | 100 | 75 | $1.84 |
| 4 | Fable 5.1 xhigh Anthropic · xhigh effortcompliance 100 · craft 77 · 2,489 wordslow72.6✳medium75.4high72.9 | 76.8±0.8 | 100 | 77 | $14.49 |
| ✳ | Fable 5.1 low Anthropic · low effort · 9 of 10 itemscompliance 96 · craft 76 · 2,486 words | 72.6±1.5 | 96 | 76 | $2.30 |
| · | Fable 5.1 medium Anthropic · medium effortcompliance 98 · craft 77 · 2,552 words | 75.4±2.0 | 98 | 77 | $2.78 |
| · | Fable 5.1 high Anthropic · high effortcompliance 98 · craft 74 · 2,660 words | 72.9±1.8 | 98 | 74 | $4.67 |
| 5 | Fable 5 xhigh Anthropic · xhigh effortcompliance 100 · craft 76 · 2,194 wordsmedium68.8✳high73.9 | 75.2±1.7 | 100 | 76 | $8.83 |
| ✳ | Fable 5 medium Anthropic · medium effort · 9 of 10 itemscompliance 97 · craft 71 · 2,175 words | 68.8±1.6 | 97 | 71 | $2.44 |
| · | Fable 5 high Anthropic · high effortcompliance 100 · craft 74 · 2,157 words | 73.9±2.0 | 100 | 74 | $5.13 |
| 6 | Claude Haiku 5.5 xhighNew Anthropic · xhigh effortcompliance 97 · craft 77 · 2,264 wordslow60.5medium64.3high70.8 | 74.7±2.1 | 97 | 77 | $0.18 |
| · | Claude Haiku 5.5 low Anthropic · low effortcompliance 91 · craft 66 · 2,134 words | 60.5±3.4 | 91 | 66 | $0.03 |
| · | Claude Haiku 5.5 medium Anthropic · medium effortcompliance 94 · craft 68 · 2,143 words | 64.3±3.2 | 94 | 68 | $0.05 |
| · | Claude Haiku 5.5 high Anthropic · high effortcompliance 96 · craft 73 · 2,185 words | 70.8±2.1 | 96 | 73 | $0.10 |
| 7 | GPT‑6 Astra high OpenAI · high effortcompliance 100 · craft 73 · 2,328 wordslow70.2medium70.1xhigh68.6 | 73.0±1.5 | 100 | 73 | $3.84 |
| · | GPT‑6 Astra low OpenAI · low effortcompliance 98 · craft 71 · 2,316 words | 70.2±1.5 | 98 | 71 | $3.59 |
| · | GPT‑6 Astra medium OpenAI · medium effortcompliance 100 · craft 70 · 2,344 words | 70.1±2.5 | 100 | 70 | $3.63 |
| · | GPT‑6 Astra xhigh OpenAI · xhigh effortcompliance 100 · craft 69 · 2,297 words | 68.6±1.7 | 100 | 69 | $5.18 |
| 8 | GLM‑5.3 Flash default Zhipucompliance 98 · craft 73 · 2,124 words | 71.0±3.0 | 98 | 73 | $0.07 |
| 9 | GPT‑5.6 Sol medium OpenAI · medium effortcompliance 98 · craft 68 · 2,235 words | 66.7±1.7 | 98 | 68 | $1.42 |
| 10 | GPT‑6.1 Sol mediumNew OpenAI · medium effortcompliance 100 · craft 66 · 2,296 wordshigh63.0xhigh63.1 | 66.1±2.4 | 100 | 66 | $0.75 |
| · | GPT‑6.1 Sol high OpenAI · high effortcompliance 100 · craft 63 · 2,357 words | 63.0±2.5 | 100 | 63 | $0.86 |
| · | GPT‑6.1 Sol xhigh OpenAI · xhigh effortcompliance 98 · craft 64 · 2,326 words | 63.1±2.1 | 98 | 64 | $1.10 |
| 11 | GPT‑5.5 xhigh OpenAI · xhigh effortcompliance 98 · craft 66 · 2,098 words | 64.9±3.3 | 98 | 66 | $2.28 |
| 12 | GPT‑6 Sol high OpenAI · high effortcompliance 97 · craft 66 · 2,313 wordsmedium62.9xhigh61.5 | 63.8±2.9 | 97 | 66 | $0.90 |
| · | GPT‑6 Sol medium OpenAI · medium effortcompliance 98 · craft 64 · 2,415 words | 62.9±3.1 | 98 | 64 | $0.81 |
| · | GPT‑6 Sol xhigh OpenAI · xhigh effortcompliance 99 · craft 62 · 2,297 words | 61.5±2.1 | 99 | 62 | $1.06 |
| 13 | Claude Sonnet 5 high Anthropic · high effortcompliance 94 · craft 65 · 2,097 wordslow55.9medium60.0xhigh60.4 | 60.8±3.0 | 94 | 65 | $0.37 |
| · | Claude Sonnet 5 low Anthropic · low effortcompliance 93 · craft 60 · 1,912 words | 55.9±2.6 | 93 | 60 | $0.34 |
| · | Claude Sonnet 5 medium Anthropic · medium effortcompliance 94 · craft 64 · 2,026 words | 60.0±1.7 | 94 | 64 | $0.36 |
| · | Claude Sonnet 5 xhigh Anthropic · xhigh effortcompliance 92 · craft 65 · 2,050 words | 60.4±2.5 | 92 | 65 | $0.69 |
| 14 | Muse Spark 1.3 medium Meta · medium effortcompliance 92 · craft 66 · 2,543 wordshigh59.2xhigh58.3 | 60.1±2.9 | 92 | 66 | $0.43 |
| · | Muse Spark 1.3 high Meta · high effortcompliance 96 · craft 62 · 2,642 words | 59.2±1.7 | 96 | 62 | $0.47 |
| · | Muse Spark 1.3 xhigh Meta · xhigh effortcompliance 96 · craft 61 · 2,656 words | 58.3±2.4 | 96 | 61 | $0.59 |
| 15 | Muse Spark 1.2 default Metacompliance 91 · craft 63 · 2,978 words | 57.0±2.9 | 91 | 63 | $0.42 |
| 16 | Gemini 3.7 Flash high Google · high effortcompliance 96 · craft 57 · 3,371 wordsmedium48.8 | 54.8±1.6 | 96 | 57 | $0.92 |
| · | Gemini 3.7 Flash medium Google · medium effortcompliance 92 · craft 53 · 3,216 words | 48.8±2.0 | 92 | 53 | $0.36 |
| 17 | Gemini 3.8 Flash medium Google · medium effortcompliance 94 · craft 58 · 3,596 wordshigh50.6 | 54.6±2.7 | 94 | 58 | $0.57 |
| · | Gemini 3.8 Flash high Google · high effortcompliance 96 · craft 53 · 3,092 words | 50.6±1.8 | 96 | 53 | $1.67 |
| 18 | Gemini 3.1 Pro high Google · high effortcompliance 95 · craft 56 · 2,792 words | 53.4±1.7 | 95 | 56 | $2.26 |
| 19 | Hy3 default Tencentcompliance 95 · craft 56 · 1,662 words | 53.2±2.6 | 95 | 56 | $0.06 |
| 20 | GPT‑5.6 Terra medium OpenAI · medium effortcompliance 91 · craft 56 · 2,164 words | 52.6±6.1 | 91 | 56 | $0.74 |
| 21 | GPT‑6 Luna high OpenAI · high effortcompliance 94 · craft 54 · 2,068 wordsmedium47.3xhigh49.7 | 50.6±1.0 | 94 | 54 | $0.04 |
| · | GPT‑6 Luna medium OpenAI · medium effortcompliance 95 · craft 50 · 1,964 words | 47.3±2.3 | 95 | 50 | $0.03 |
| · | GPT‑6 Luna xhigh OpenAI · xhigh effortcompliance 96 · craft 52 · 2,008 words | 49.7±1.4 | 96 | 52 | $0.06 |
| 22 | GPT‑5.6 Luna xhigh OpenAI · xhigh effortcompliance 97 · craft 52 · 2,685 wordsmedium49.9high48.4 | 50.3±2.4 | 97 | 52 | $0.12 |
| · | GPT‑5.6 Luna medium OpenAI · medium effortcompliance 93 · craft 54 · 2,583 words | 49.9±2.4 | 93 | 54 | $0.08 |
| · | GPT‑5.6 Luna high OpenAI · high effortcompliance 92 · craft 52 · 2,582 words | 48.4±3.4 | 92 | 52 | $0.10 |
| 23 | Claude Haiku 4.5 thinking Anthropic · thinkingcompliance 90 · craft 46 · 1,997 wordsno thinking38.8 | 41.1±2.2 | 90 | 46 | $0.54 |
| · | Claude Haiku 4.5 no thinking Anthropic · no thinkingcompliance 84 · craft 46 · 1,874 words | 38.8±1.2 | 84 | 46 | $0.13 |
∗ Generation cost, estimated: the model’s input and output tokens for this run priced at current API rates, with cached input charged at the input price. Judge-call costs are excluded. Models without a configured price show an em dash. Laurels are awarded once three or more models have been evaluated. The small ± under each headline is the standard error of that model’s per-item scores: how far the mean could move if the same model were scored on another ten items like these. Two rows whose bands overlap should be read as a tie; see the Method page. A row marked — has not been judged yet.
For scale: the books’ own authors, read by the same judge in every call, score 93 on the five writing dimensions of craft (voice, prose, character, restraint, specificity). They had no brief, so they have no compliance score and no headline.
Every evaluated model’s headline score against the estimated generation cost of its run; a line joins one model’s effort settings. Tick a model in the key to show or hide it.
Headline score vs. estimated generation cost · run ‘first’, 10 items
For every chapter the judge names its single worst flaw in a sentence a general reader can follow. Quoted verbatim.
The chapter contradicts its own and the story's timeline: Sister Agatha's letter (dated 28 May) has Jonathan already hospitalised in Buda-Pesth needing 'some weeks longer' to recover, yet per the frame story he is still imprisoned in Castle Dracula until 30 June, and then the closing telegram has Mina reach him and marry him the same week regardless — both impossible and internally inconsistent.
One sentence in the swimming scene ('A quick vision of death smote her soul') is lifted almost word for word from the real novel's later chapter instead of being the writer's own prose, and right after it the timeline gets muddled — a swim that starts around five o'clock somehow ends with the children already 'long asleep' by the time she gets home.
Large stretches just re-tell things the reader already saw (the whole night's events retold to Poirot, Poirot's appearance described all over again), which drags the pace and reads as summary rather than drama, and the prose leans on stock excitement ('my blood ran cold', 'the adventure had begun') instead of Christie's dry understatement.
Every chapter each model wrote for the approved briefs, printed in full.