Research

The products on this site grow out of research into how large language models behave as creative and agentic systems. This page indexes that work: the question each project asks, what we found, and where to dig into the methodology and data. Results are dated snapshots, not live leaderboards, and every claim links back to a repository where the runs can be reproduced.

LLM Creative Writing Benchmark

Original study 2025 · longitudinal re-run June 2026 · ongoing

When you ask a model for the same story ten times, how differently does it write? We send one science-fiction prompt to each model ten times and measure how much the wording, ideas, vocabulary, and character names vary. The June 2026 re-run covers Gemini 3, Claude Opus 4.8, Sonnet and Fable 5, and GPT-5.5, and finds the old "Elara" naming habit has split sharply by vendor.

Human Masterworks vs AI Fiction

Masters corpus · 26 books scored · July 2026 · ongoing

How does AI-generated fiction compare to the real thing? We scored 26 canonical, public-domain masterworks on the same lexical, structural, and narrative-dynamics tooling we point at model output, building a human baseline for creativity and diversity rather than for AI detection. The result is a reference band: the range real literature occupies, so AI prose can be graded against books rather than against a guess at what "good" looks like.

Open-Source AI Agent Reviews

Started 2026 · ongoing

A structured, code-level review programme covering open-source AI agent projects: general-purpose assistants, coding agents, and agent frameworks. Each project is reviewed at a pinned commit, with findings backed by evidence from the source code, so the assessments stay checkable as the projects evolve. The reviews themselves are deliberately not a ranking, but a way to understand how agent harnesses are actually built; a separate, model-controlled harness benchmark (scored with the official SWE-bench scorer) now runs alongside them. That same understanding shapes our own agentic software.