Diffeomorphic Stablity under Ricci Curvature Lower Bound