Research
The products on this site grow out of research into how large language
models behave as creative and agentic systems. This page indexes that
work: the question each project asks, what we found, and where to dig
into the methodology and data. Results are dated snapshots, not live
leaderboards, and every claim links back to a repository where the runs
can be reproduced.
Original study 2025 · longitudinal re-run June 2026 · ongoing
When you ask a model for the same story ten times, how differently does
it write? We send one science-fiction prompt to each model ten times
and measure how much the wording, ideas, vocabulary, and character
names vary. The June 2026 re-run covers Gemini 3, Claude Opus 4.8,
Sonnet and Fable 5, and GPT-5.5, and finds the old "Elara" naming habit
has split sharply by vendor.
Read the full study →
Masters corpus · 26 books scored · July 2026 · ongoing
How does AI-generated fiction compare to the real thing? We scored 26
canonical, public-domain masterworks on the same lexical, structural, and
narrative-dynamics tooling we point at model output, building a human
baseline for creativity and diversity rather than for AI detection. The
result is a reference band: the range real literature occupies, so AI
prose can be graded against books rather than against a guess at what
"good" looks like.
See the study →
Open-Source AI Agent Reviews
Started 2026 · ongoing
A structured, code-level review programme covering open-source AI agent
projects: general-purpose assistants, coding agents, and agent
frameworks. Each project is reviewed at a pinned commit, with findings
backed by evidence from the source code, so the assessments stay
checkable as the projects evolve. The reviews themselves are deliberately
not a ranking, but a way to understand how agent harnesses are actually
built; a separate, model-controlled harness benchmark (scored with the
official SWE-bench scorer) now runs alongside them. That same
understanding shapes our own agentic software.
GitHub repo →