Yeah, they had chemistry all right...as enemies, and I just don't think that naturally transfers to romantic chemistry. I loved the way they interacted as enemies...obviously they were similar in ways that helped make these interactions so fascinating. That doesn't mean them becoming lovers makes a lick of sense. It's like Batman and The Joker. In many ways, they are so alike. They're both people who operate outside the law and follow their own instincts rather than societal norms as they're motivated by psychological problems/past traumas. They're both stubborn, obsessive, and passionate, but the difference is, one's actions are for justice and the other's are for chaos. So they're obviously made for each other...that doesn't mean it would make sense for them to start having sex.
Funny you mention Batman, since what you've described has its own Television Trope and it's called
Dating Catwoman.
Spike as a love interest for Buffy, though, was one contrivance too many. To me, it just smacked of the writers running out of ideas. 'Well, we've run through two Buffy boyfriends. We don't have anything else for Spike to do, but we want her in a couple and we want to keep him around. Two birds, one stone, let's just put them together!'
Actually, Joss Whedon first toyed with the idea during season 4, and it might have been the actors who first put the idea into his head. In an early S4 interview with Sarah Michelle Gellar, she said Joss had asked her who could be Buffy's new boyfriend now that Angel was gone, and she said that she wanted it to be Spike, but Joss said that maybe having her date another vampire would be too much. James Marsters said that he always played Spike's interest in Buffy as being somewhat sexual and that when he was told Spike would be revealed to be in love with Buffy, he told Joss that he had been hoping for that but was afraid it would be too 'presumptuous'. By "Something Blue", the writers seemed to be toying with the idea, and by season 5 they had decided to have a Buffy/Spike storyline (though I don't know how much was decided already) - and that's when they had the Buffy/Riley breakup, which, if you remember, Spike had some involvement in, reinforcing Riley's insecurities and the idea that Buffy could never be fully satisfied with a normal human who didn't have something of a 'monster' in him. So, it's not like they went through two boyfriends as you said, it's that they set up a Riley/Spike contrast since the beginning of season 5.
Also, the season 5 premiere "Buffy vs Dracula" was full of hints about Buffy's dark side and her attraction to vampires (and the strain it put on her relationship with Riley).
Then, the writers actually expected us to really seriously believe Buffy might have been on to something with that line about hiding their true feelings through hostility? Absurd. I just can't understand how two people who hated each other so much could possibly end up having any kind of relationship, even an unhealthy one. The idea started as a joke, and should have stayed that way. It never deserved to be anything more. I didn't mind when he was in love with her and she was rejecting him. It was amusing when she was trying to figure out why he liked her and said, "I beat him up a lot, maybe to him that's like second base".
And nobody in the room thought or dared to ask her: "So... if that's what you think, why do you beat him up so much, Buffy?"
Also, some other interesting questions, like why did she get so freaked out when he first said he was in love with her that she revoked the invitation to the Summers house for the first and only time (she never bothered to do that since the season 2 finale) - she was apparently more scared of him being in love with her than she was of him trying to kill her (as in early season 4)?
Sorry, but I never ever bought that she was just "disgusted" by the idea of the two of them together. The way Buffy's behavior with Spike was written and portrayed, especially season 4-5, with overstated hostility, a lot of unnecessary aggression and too many angry protestations of disgust and hate, you just couldn't help thinking "she doth protest too much". Of course she wouldn't ever consciously contemplate the idea under the normal circumstances. But the show put her in the circumstances and psychological state where it was believable, IMO. And she was still deeply tormented and disgusted by herself. It's not like they made it into a fluffy romance a la "Something Blue", which was something that could have only happened under the spell. I understand that a lot of people have fun with the idea of a 'bad guy' crushing on the heroine but prefer the heroine to retain her proper and savory image. But really, that's more of a stereotype than you realize, and a very old one - it goes back to the 18th century sentimental and Gothic novels where the chaste heroines were pursued by lustful villains but never gave up their virtue.
Then, the writers actually expected us to really seriously believe Buffy might have been on to something with that line about hiding their true feelings through hostility? Absurd.
Unless you remember what Buffy (at that point fully sane and not under any spell) told Willow earlier in the same episode - while discussing Riley:
Buffy: I don't know. I really like being around him, you know? And I think he cares about me.. but.. I just.. feel like something's missing.
Willow: He's not making you miserable?
Buffy: Exactly. Riley seems so solid. Like he wouldn't cause me heartache.
Willow: (Fake worry) Get out. Get out while there's still time.
Buffy: I know.. I have to get away from that bad boy thing. There's no good there. Seeing Angel in LA.. even for five minutes.. hello to the pain.
Willow: The pain is not a friend.
Buffy: But I can't help thinking — isn't that where the fire comes from? Can a nice, safe relationship be that intense? I know it's nuts, but.. part of me believes that real love and passion have to go hand in hand with pain and fighting.
(Suddenly a vampire jumps out from behind a bush. Buffy stakes him without so much as looking the other way. He crumbles to dust and she and Willow continue on their way)
Buffy: I wonder where I get that from.
Is that messed up? Yes. Absurd? For people with healthier experiences and outlook on love/sex, probably. But there are plenty of people who keep being drawn to dysfunctional and messed up relationships, and Buffy always seemed a bit like that.
That episode was full of 'absurd' things that seem like foreshadowing of the later storyline. Like broken-hearted Willow becoming vindictive, and starting using magic too much to fix all her problems.
Even season 1, which was the weakest IMO (I don't hate season 4 like many do). 
