We consider the movement of the set of maximum points of the solution of the heat equation in the exterior domain of a ball, under the Neumann boundary condition. We prove that the set of the maximum points of the solution tends to one point, which is represented by the initial data explicitly, as the time tends to infinity. Furthermore we consider the movement of the set of maximum points under the Dirichlet boundary condition.