There is a growing interest among both mathematicians and AI researchers to apply AI models to mathematical questions. Unfortunately, many AI models are black boxes. Experts understand poorly both the process by which they arrive at their predictions and how to interpret the predictions they give. In this largely expository talk, I will survey some recent work in mechanistic interpretability, an empirically-driven field whose goal is to extract algorithms from trained AI models. After reporting on findings from an undergraduate research project I co-directed (with C. Ashley) on mechanistic interpretation of several different AI model solutions to the word problem in Coxeter groups, I will suggest promising avenues for future research and comment on the challenges involved in pursuing them.