Harry isn't as hurt by some of the normally arsey things that Draco does (if that makes sense). He grew a thick skin to them while they were still dating. But he does intend to hold Draco to the fact that his words very deeply hurt Harry, intentionally or not.