I didn't think soulless Sam was out of character. It was a different side of his character, and I personally thought Jared did a great job playing Sam like that.
Just because people like certain things about a story or a character, it doesn't mean that all we should ever get are cop-outs and reset buttons. That's my main gripe with Dean's current character arc, just like it's my main gripe with DS9's Sacrifice of Angels.
I used to say that the Prophets thing was a deus ex machina, but SF Debris is right: it's not. The show was building up their involvement, so it didn't come out of nowhere, but it was a cop-out imho. You could have had the Dominion win that bit of the war and still not throw overboard the things that audiences liked about the show's storylines and characters.
One of the greatest thing in scifi is the "what if" question, a sense of real danger, of not knowing what's going to come next for characters we love. The reset button takes all that excitement away, and it's cheap, lazy story-writing. Sam's soul-search actually had consequences that are palpable even now.
Dean being a demon for 3 episodes and then being magically cured...that's Sacrifice of Angels all over again, and I sure hope TPTB have a trick up their sleeve for compensating for that major letdown. I actually liked him better that way, even if he was kind of useless.