Neural networks have become a powerful tool in various domains of scientific research and industrial applications. However, the fundamental working principles of neural architectures still lacks of a solid theoretical framework, which prevents a true understanding of their information processing features and the associated emergence of collective properties thus making the realization of optimized models and algorithm a challenging problem. This talk provides a mathematical perspective on the theory of neural networks. After a brief introduction of the fundamental concepts of statistical mechanics and mean-field models of spin-glasses, we will move to the analysis of the so-called Dense Associative Memories. We will show how to frame the statical-mechanics description of these models with the theory of partial differential equations in order to gain a deeper comprehension of the system's subtle mechanisms.