Pretraining Safety w/ Ethan Roland

What if the safest AI models weren't built by adding guardrails after training, but by shaping what gets learned in the first place? Ethan Roland, senior alignment researcher at AE Studio and first author on an ICML 2026 spotlight paper, joins Jacob to talk about gradient routing, a technique that routes dangerous capabilities into isolated parts of a model's architecture where they can be locked or removed entirely. They get into the absorption effect, KYC-style access control frameworks, and what it would actually take for frontier labs to adopt this kind of work before it's needed rather than after.

Check out the video version of this podcast on the Kairos.fm YouTube channel!

Chapters

Links

• Ethan’s website
• Paper landing page
• Anthropic’s press release and blogpost

Relevant Prior Works

• Preprint - Gradient Routing: Masking Gradients to Localize Computation in Neural Networks
• Preprint - Beyond Data Filtering: Knowledge Localization for Capability Removal in LLMs
• ICLR paper and webpage - Deep Ignorance: Filtering Pretraining Data Builds Tamper-Resistant Safeguards into Open-Weight LLMs
• ICLR paper - The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks
• ICLR paper - Self-Improvement in Language Models: The Sharpening Mechanism

CBRN(e)

• Wikipedia article - CBRN defense
• NTI tutorials on bioweapons and nuclear testing

Other Sources

• Exploring Language Models substack article - A Visual Guide to Mixture of Experts (MoE)
• ICLR paper - Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
• Swift blogpost - The KYC process explained
• FATF recommendations on KYC