Thank you!

Harry probably should have flipped over the portraits at the beginning, but, of course, he had no way of knowing what they were then.

Draco had fewer memories to steal, and he's tied into the maze, so the portraits mostly concentrated on Harry. He lost a lot more memories, and a lot faster.

Harry will try ignoring the existence of the problem for a while longer. And if that doesn't work, there's always good old denial.