Our Projects
Consumer Behaviour, Market Dynamics, and Welfare Measurement
Vicky is Senior Research Manager at Ambitious Impact (AIM) leading their animal welfare research to identify new charity ideas to be launched through its Charity Entrepreneurship Incubation Program and mentoring animal welfare focused participants of its Research Program.
Before joining the AIM team, Vicky went through the CE Incubation Program. Prior to this, she was a research intern for AIM, primarily completing the priority country research for our recommended animal charities. She did this internship alongside her Maths with Actuarial Science degree at the University of Southampton, where she helped to found an effective altruism society. Vicky went vegan and became interested in effective altruism in 2016, after reading works by Peter Singer.
Vicky is excited about projects explore some of the biggest uncertainties in animal welfare research: how consumer behaviour translates into real-world changes in animal product supply, whether plant-based alternatives truly substitute for meat or simply add to consumption, and how to systematically compare the intensity of suffering across species and pain types. Together, they aim to close critical evidence gaps that shape the effectiveness of advocacy, industry change, and charity strategy.
Synthetic Signatures: Detecting and Classifying AI-Generated Genomes
Aaron Maiwald is a Dphil at the University of Oxford in statistics, focusing on the intersection of AI and biology, with with the ultimate goal of pandemic prevention. His work is supported by the Berkeley Existential Risk Initiative.
AI biodesign tools are enabling us to generate novel organisms, outpacing natural evolution. For example, the genomic language model EVO-2 has been used to generate entire, functional bacteriophage genomes with low similarity to natural counterparts. While bacteriophages are harmless to humans, it is plausible that models like EVO can soon be used to generate human-infecting viral genomes that are highly dissimilar from known pathogens. For DNA-synthesis screening and metagenomic surveillance to detect such threats, we need tools that can identify AI-designed sequences. In this project, we will aim to create a dataset of AI-generated sequences and build classifiers to distinguish them from natural and traditionally engineered sequences. We might extend this to not only detect if AI design is present but also detect which AI tool was used.
LLMs as architects for multi-agent systems, and exploring information flows in LLMs
I'm a DPhil student in Machine Learning in the FLAIR lab at the University of Oxford supervised by Jakob Foerster and funded through the AIMS CDT by the EPSRC. My research focuses on scalable methods for reducing risks in advanced AI systems, exploring AI Safety, Interpretability, and Multi-Agent Systems all through the unifying lens of scale.
Previously, I was a Research Scientist Intern at Spotify and worked with the UK AI Security Institute (AISI) on their Bounty Programme investigating automated design of agentic systems. I was also the founding Research Scientist at Convergence (acquired by Salesforce), contributing to Proxy, a state-of-the-art multimodal web agent with 100k+ users, and held senior engineering roles at Pynea and Artera, leading teams and shipping ML innovations.
LLMs as architects for multi-agent systems
This project explores the use of large language models (LLMs) as automated architects for complex multi-agent systems, with cybersecurity as a test domain. Using the CyberAgentBreeder framework to design Python-based scaffolds for Capture-the-Flag challenges, the research evaluates the quality, novelty, and sophistication of agent architectures generated by different models. Building on early evidence that state-of-the-art LLMs can propose fundamentally new designs, the project aims to determine how model scale and reasoning ability shape the complexity of resulting systems, including features like hierarchical team structures, specialized roles, and advanced communication protocols.
Exploring Information Flows in LLMS
As transformer-based large language models (LLMs) scale to million-token contexts, understanding how information deeply embedded in the context flows through the neural network to the output becomes critical. This project investigates the interplay of three emerging failure modes in long-context LLMs - over-squashing, where crucial signals are compressed into indistinguishable representations; over-smoothing, where representations across tokens become homogenized; and under-reaching, where information fails to propagate far enough.
Using a novel interpretability framework based on sparse attention (work under submission), we can prune the attention pattern to identify key information flow routes, without incurring the prohibitive $O(T^2)$ memory overhead of dense attention interpretability.
Aquila is a Research Project Manager at GovAI, ensuring the organisation's initiatives and research programs run smoothly. She works closely with the Director of Research and Policy to execute strategic priorities and ensure programs support the research mission. Before joining GovAI, she worked as an engineer on major infrastructure projects. She holds BA and MEng from the University of Cambridge.
How quickly are embodied AI Systems being adopted?
This project examines the pace of embodied AI adoption by asking what share of large U.S. warehouses will use autonomous mobile robots (AMRs) for inventory management by 2028 and 2031. Warehouses are among the first major testbeds for embodied AI, with companies like Amazon, Walmart, and DHL already deploying robots to improve efficiency and reduce labor costs. Tracking adoption rates in this sector offers an early window into how quickly AI systems that act in the physical world are scaling, what barriers and risks shape their deployment, and what this means for the broader trajectory of AI integration across the economy.
Matthew is a DPhil student working on foundational research for technical AI safety, including agent foundations and singular learning theory. Matthew previously worked on understanding goal misgeneralisation at Krueger AI Safety Lab, applying singular learning theory with Timaeus, and the foundations of reward learning at CHAI. Matthew is excited to help aspiring researchers develop their research skills and contribute to technical AI safety.
Invariance-aware diverse reward model ensembles
We want powerful AI models to internalise human goals. Modern approaches involve (either explicitly or implicitly) learning a reward function from human data generated based on these goals, then extrapolating that reward function. However, often there are multiple plausible but conflicting ways to extrapolate the goals from the given data. If we were able to detect this kind of multiplicity at the reward modelling stage, we could avoid catastrophic misgeneralisation. A standard method for detecting and addressing ambiguity in supervised learning is by training a diverse ensemble of models (rather than a single model). This project seeks to extend such methods to the domain of reward learning, accounting for the unique invariance structures of RL.
Migration Risk in Catastrophic Food Failures: Testing Theory, Learning from Crises, Building Resilience
Noah Wescombe is a Policy Lead at ALLFED where he and Senior Specialist in Climate Resilience at Cadlas. Previously he was a research advisor at the Cambridge Existential Risks Initiative and a research fellow at the Stanford Existential Risks Imitative. He received his masters in philosophy at Oxford University.
This project investigates how global catastrophic food failures (GCFFs) could drive migration. While theories of migration under stress are well established, there is limited empirical testing of how these models hold up in real crises. The project will bridge this gap by applying migration theory to historical food crises, incorporating behavioural and cultural perspectives, and exploring how cascading risks like conflict or governance failures interact with food-related shocks. The aim is to produce policy-relevant insights that can strengthen resilience and inform responses to crisis-driven migration.