Our Projects – Michaelmas 2026
Regulating with AI Agents: A Map and an Agenda
AI capabilities are advancing rapidly and unpredictably, creating risks ranging from sophisticated cyberattacks to misaligned systems pursuing goals their developers did not intend. At the same time, regulators face difficulties monitoring increasingly complex and widely deployed AI systems. This project examines whether AI agents could themselves support regulation. It will map the governance functions agents could perform, consider how these functions could be operationalised and by whom, evaluate the associated practical and ethical issues, and develop an agenda for future research on regulating with AI agents.
Huw is a researcher specialising in AI governance, with a particular focus on international and comparative AI governance and the institutions and standards involved in governing advanced AI. His background spans academic research and practical AI policy, including previous work for the UK Government.
Huw Roberts
Forecasting Swarm Size
Efforts to quantify AI loss-of-control risk depend partly on the “size” of rogue AI deployments: how much covert project work they can complete before developers detect them and mount resistance. This project will develop a more empirically grounded measure of swarm size by surveying relevant benchmarks and evaluations, estimating swarm size across recent incidents, testing for trends over time, and investigating how swarm size relates to a rogue deployment’s ability to acquire resources or inflict mass harm.
Paolo is a researcher whose work focuses on quantitative approaches to AI risk and governance. His previous research has included modelling AI regulatory incentives and forecasting gaps between growing AI capabilities and the ability of evaluations to identify emerging risks.AI capabilities are advancing rapidly and unpredictably, creating risks ranging from sophisticated cyberattacks to misaligned systems pursuing goals their developers did not intend. At the same time, regulators face difficulties monitoring increasingly complex and widely deployed AI systems. This project examines whether AI agents could themselves support regulation. It will map the governance functions agents could perform, consider how these functions could be operationalised and by whom, evaluate the associated practical and ethical issues, and develop an agenda for future research on regulating with AI agents.
Paolo
Corporate Incentives and Gradual Disempowerment
AI is likely to empower corporations more than individual humans. Because profits are an imperfect proxy for human welfare, increasingly automated profit-maximising corporations could contribute to gradual disempowerment. This project will investigate whether this risk is plausible and through which mechanisms, focusing on externalities, endogenous preferences, and the ways AI could weaken the influence of workers, governments and consumers. The project will synthesise these mechanisms and their assumptions and, if time allows, develop or adapt a stylised formal model.
Matt is based at the University of Warwick. His research sits at the intersection of economics and other social sciences, with a particular interest in econometrics.
Matt
Which Research Counts as Dual-Use?
Policy documents increasingly use the term “dual-use research”, but the criteria used to define it are rarely tested on actual research papers. This project asks whether existing criteria can be applied consistently and what they miss. The participant will build a dual-use screening checklist from existing criteria, apply it to around 100 recent papers and preprints from one area of the life sciences, and analyse where the checklist produces clear, conflicting or incomplete results.
Mira is a pharmacist and researcher with a DPhil in Clinical Medicine from Oxford. Her research background includes medicinal chemistry, drug discovery and drug delivery, alongside an interest in biosecurity and dual-use biological research.
Mira
Investigating the Impact of the Language of Reasoning
When a large language model “thinks” before answering, does the language it thinks in change how well it reasons? Modern reasoning models produce long chains of thought that are overwhelmingly in English, regardless of the input language. This project will test whether reasoning language affects capability across mathematical and logical reasoning, factual recall and culturally grounded judgement, investigate the source of any performance gaps, and test interventions designed to close them.
Ej is an NLP PhD researcher at the University of Cambridge’s Language Technology Lab. His research focuses on multilingual language models, including cross-lingual representations and model calibration across languages.
Land Value Taxation in the UK
Land value taxes are often argued to be more economically efficient than taxes on productive activities because taxing land does not reduce its underlying supply. Despite this theoretical case, land value taxation remains rare and faces significant political and implementation challenges. This project will investigate the main hurdles to a shift towards land value taxation in the UK and, where possible, ways to address them, drawing on previous literature, interviews with politicians and other policy-relevant actors, and lessons from previous implementation efforts.
Martin is a postdoctoral researcher in the Department of Government at Uppsala University. He works in the interdisciplinary field of Politics, Philosophy and Economics, with a special focus on the political and economic theory of Georgism. His PhD thesis, Land & Liberty, was named best thesis of 2022–24 by the Swedish Political Science Association. He is also involved in Georgist movement-building and organising the Geoist Network Conference.
Warning shots are events that signal larger risks before their most severe consequences materialise, but some generate sustained attention and action while others do not. This project will use historical case studies to investigate why. Participants will examine the communication strategies surrounding warning shots and the political, cultural and institutional environments that shaped their effects. The project may also refine the definition and taxonomy of warning shots and use its historical findings to develop evidence-based recommendations for communication following future warning shots.
Evan has a background spanning biotechnology and the history of science, including genome-editing research and an MPhil in History and Philosophy of Science and Medicine at Cambridge. Isha is a Cambridge medical student and biochemistry graduate whose recent work has included biosecurity and AI-safety research.
Warning Shot History and Political Communication in AI Safety
