Geometric deep learning focuses on machine learning problems with known symmetries. For example, learning problems on molecules are often invariant to permutation and rigid motions of the molecule’s atoms. Typical approaches for these problems rely on invariant embeddings of the objects of interest into Euclidean space. In this talk we will focus on two basic questions: (a) Injectivity: is the embedding injective, and (b) Bi-Lipschitzness: Does the embedding preserve distances, at least approximately? We will explain the motivation for these questions, present results which point to the advantage of sorting based methods over methods based on summation, and discuss open questions and empirical aspects.