Hi, I’m Ryan, a Founding Engineer at Composo AI in London. My current work focuses on making automated evaluation reliable enough to deploy, through LLM-as-a-judge calibration, uncertainty quantification, and classifier fine-tuning for guardrails.
My interest is AI safety, and specifically AI control: whether the judges and monitors we rely on can be trusted when a capable agent is trying to get past them. I’m currently working on this as part of SPAR’s In-the-Wild AI Control project.
I use this site to write up papers I’ve worked on and think through research in progress.
Posts
-
Decomposing LLM-Judge Uncertainty to Target Expert Labels
Which items should a human actually label? Only the ones where the judge is ignorant, not the ones where the experts disagree.
-
Quantifying blind spots of LLM Evaluators
When an LLM judge is wrong, does its own resample disagreement know?
-
Making LLM judges more reliable without fine-tuning
Improving RewardBench 2 performance by 13.5pp; accepted at the ICML 2026 Workshop "Statistical Frameworks for Uncertainty in Agentic Systems"