Graph Neural Networks (GNNs) have emerged as powerful architectures for processing relational data and executing complex algorithmic tasks. However, understanding how these models compute remains a fundamental challenge. In this talk, I will discuss two complementary approaches to interpreting GNNs, moving from post-hoc explanations of individual predictions to identifying the internal computations learned by the network. First, we study GNN explanation methods that identify important edges or subgraphs for a prediction, and establish theoretical connections between several popular methods (e.g., perturbation-based) and edge gradients, which are simple and efficient to compute. Next, I will present Mechanistic Interpretability for Neural Algorithmic Reasoning (MINAR), an approach for identifying internal circuits responsible for algorithmic computation in GNNs. Focusing on GNNs trained on classical graph algorithms, we show how algorithmic circuits form and are pruned during training and how circuit components are shared across related algorithmic tasks.