Christopher Potts - Creating and detecting subliminal learning effects on demand
September 1, 2026
Abstract
This talk will argue for a data-centric approach to interpretability. I will do this through the lens of subliminal learning, in which language models acquire behaviors from examples that seem unrelated to those behaviors (Cloud, Le, et al. 2025). I will first focus on using Data Policy Gradients (DPG; Thrush et al. 2026) to generate these effects on demand. DPG is a framework for synthesizing examples that, when used as training data, cause the target model to do well on a chosen differentiable metric, including surprising metrics like embedding a QR code pattern in the model weights. DPG creates a setting in which we can study the causal links between examples and behaviors, for subliminal effects as well as more obvious ones. I will then turn to detecting subliminal effects on demand, across a range of adversarial settings, using soft-prompt learning (Hu and Potts 2026). Finally, I will present a case study of sycophancy in OLMo. In Blank et al. 2026, we trace the relevant sycophantic behaviors specifically to the DPO phase of OLMo training, where they emerge as a subliminal effect stemming from the relationship between two models used to generate preference pairs. This analysis gives us fine-grained control over the sycophantic behavior itself. Taken together, these perspectives on subliminal learning provide a rich picture of the origin and nature of the effects, and they show that, by focusing on the relationship between data and model behaviors, we obtain real power to detect, characterize, and modulate the effects before they become embedded in model representations.
Cloud, Alex; Minh Le; James Chua; Jan Betley; Anna Sztyber-Betley; Jacob Hilton; Samuel Marks; and Owain Evans. 2025. Subliminal Learning: Language models transmit behavioral traits via hidden signals in data. https://arxiv.org/abs/2507.14805
Hu, Nathan and Christopher Potts. 2026. Verbalizing hidden fine-tuning effects with text optimization. To appear on arXiv!
Thrush, Tristan; Sung Min Park; Herman Brunborg; Luke Bailey; Marcel Roed; Neil Band; Christopher Potts; and Tatsunori Hashimoto. 2026. Synthetic data for any differentiable target. To appear in Proceedings of CoLM. https://arxiv.org/abs/2604.08423
Blank, Camila; Zhuofan Ying; Christopher Potts; Peter Hase; and Jing Huang. 2026. Sycophantic agreement transfers with neutral data via contrastive preference optimization. To appear on arXiv and at the CoLM Actionable Interpretability Workshop!