In join work with Manon Parent we have investigated how small the $L^2$-norm of an exponential polynomial can get if we fix the norm of the its coefficient vector. We prove lower and upper bounds of this minimum in terms of the degree of the polynomial and we apply our methods to a variant of a problem of Hilbert. We prove that on any given interval the infimum of the $L^2$-norm of exponential polynomials with integer coefficients on this interval is 0.