After an introduction to (formal) deformation quantization of Poisson manifolds, focusing on the deformation theory of associative algebras following Gerstenhaber, I will introduce constraint algebras as a way to study (symmetry) reduction in this setting. Constraint algebras encode the additional structure on the algebra of functions on a manifold needed for reduction and their deformations yield (formal) quantizations compatible with reduction. I will discuss the deformation theory of these constraint algebras and present some results about an adapted Hochschild-Kostant-Rosenberg Theorem computing the cohomology controlling this deformation theory.