Meet J Rosser

I'm a DPhil student in Machine Learning in the FLAIR lab at the University of Oxford supervised by Jakob Foerster and funded through the AIMS CDT by the EPSRC. My research focuses on scalable methods for reducing risks in advanced AI systems, exploring AI Safety, Interpretability, and Multi-Agent Systems all through the unifying lens of scale.

Previously, I was a Research Scientist Intern at Spotify and worked with the UK AI Security Institute (AISI) on their Bounty Programme investigating automated design of agentic systems. I was also the founding Research Scientist at Convergence (acquired by Salesforce), contributing to Proxy, a state-of-the-art multimodal web agent with 100k+ users, and held senior engineering roles at Pynea and Artera, leading teams and shipping ML innovations.

I live in London and am a member of LISA (London Initiative for Safe AI)! In my spare time I enjoy playing the trumpet in funk bands, running bouldering socials, and skateboarding.

My creds

  • I’ve written a few cool papers

  • I’ve been in the founding team of a few cool startups

  • I’ve worked with awesome people at AISI, Spotify, UCL etc.

Why work with me?

  • I like to think I’m passionate, friendly and approachable! 

  • I am really excited about mentorship, I love teaching!

  • We can work at your pace, I understand how busy Michaelmas can be!

What will you get out of this?

  • Each project below would make a really cool paper - if you are considering academia (e.g. a Masters/PhD) this would be a great way to find out if you like this kind of work!

  • Each project brief I’ve taken the time to de-risk e.g the project will likely go well and we’ll see something cool!

  • I’ve chosen the briefs so that they are scalable from a quick 4 page workshop paper to a 9 page main track conference paper!

Exploring Information Flow in LLMs

[ Mechanistic Interpretability ] [ Geometric Deep Learning ]

Estimated project cost: $0 if BYOC (otherwise $200)

As transformer-based large language models (LLMs) scale to million-token contexts, understanding how information deeply embedded in the context flows through the neural network to the output becomes critical. This project investigates the interplay of three emerging failure modes in long-context LLMs - over-squashing, where crucial signals are compressed into indistinguishable representations; over-smoothing, where representations across tokens become homogenized; and under-reaching, where information fails to propagate far enough.

Using a novel interpretability framework based on sparse attention (work under submission), we can prune the attention pattern to identify key information flow routes, without incurring the prohibitive $O(T^2)$ memory overhead of dense attention interpretability.

This project would be an especially great fit for:

  • STEM students or those who know degree-level maths

  • People excited about Geometric Deep Learning

  • Good programming skills

  • [Bonus] Experience with JAX or torch