They really could have. They were both young and stupid.

Draco wants an intense partner focused on him. He's much more interested in the glimpses he caught of Harry acting like that than he is in the mild-mannered one who confronts him most of the time.

And thank you!