In this talk, we explore (i) how we can use pre-existing knowledge (e.g., physics) to improve AI systems, and (ii) how we can possible extract some knowledge from AI systems. On the first topic, we present a novel physics-enhanced deep learning hybrid method, namely CondensNet, for resolving cloud physics in general circulation models. On the second topic, we present some results on the use of explainable AI for Earth System applications in the context of extreme events. We conclude with an overall perspective bridging the two topics and grounded in the digital twin paradigm.