How ready are current AI safety reporting systems to provide early warning of critical AI risks?
About me
I'm Paolo, a PhD candidate at Teesside University supervised by Professor The Anh Han. My research studies the dynamics of tech races and the effective design of early warning systems for AI risk. I have built software tools as part of Modeling Cooperation, including an interactive web app to explore tech race scenarios and an app used to facilitate the Intelligence Rising workshops that teaches ideas in AI governance to key decision makers.
Who is this for
You'll be an especially good fit if you're excited about looking at data on the strength of our early warning system for AI risks and want to think about where this might go in the next few years. I've made sure to select a project that is doable for students across a wide range of experience, so it's great for testing your fit for this kind of research.
What to expect
As a mentor, I've worked to provide a well-scoped project that I'm confident can succeed. As someone who has done independent and academic research, I'm sensitive to when projects can feel dull or have unexpected results. I find students do best when they're honest about how they feel about the project, so that the scope can be adapted to fit their interests.
As part of the team, you'll get to actively shape what the project looks like. At a minimum, we'll deliver a dataset and a blog post writing up our findings. If there is interest, this work can be extended into a conference paper or into an interactive website so that others can more easily find and use the dataset. By the end of the project, you will hopefully know something about the state of early warning systems for AI that no one else knows yet.
Short Project description
Research Question: How ready are current AI safety reporting systems to provide early warning of critical AI risks?
AI capabilities are assessed more thoroughly after deployment using evaluation methods that didn’t exist while the model was being developed. As AI systems grow more capable, will our evaluation infrastructure keep up?
To assess the above question, we’ll create a dataset that tracks key metrics from frontier AI company safety reports alongside the broader literature on evaluating AI systems for dangerous capabilities. The dataset will track risk thresholds labs identify as critical, evaluation coverage across risk domains, methodology evaluation, and timing patterns. We’ll analyse the data as we go to answer some of the following questions: (i) what risk thresholds do organisations identify as concerning? (ii) On current trends, how prepared will AI safety reporting systems be to handle emerging risks over 2026-2030?
This will be the first quantitative analysis of AI early warning system readiness, helping to inform policy debates around the need for mandatory safety auditing for frontier AI companies.