Interpretability methods are commonly evaluated through empirical plausibility, yet visually convincing explanations need not provide meaningful information about a model. This raises two complementary mathematical questions: which properties should an explanation satisfy, and to what extent can such properties be computed or verified? In this talk, I will first discuss an approach to interpretability for which rigorous guarantees can be established, including conditions ensuring consistency with the original data. I will then turn to fundamental computational limitations and their implications for interpretability, including the distinction between analog and digital computational models. Taken together, these results suggest that mathematically meaningful notions of interpretability must account not only for the model and the explanation method, but also for the underlying computational paradigm and hardware on which they are realized.