Yeah, I have to go with the time-skip as well, but the beginning of the end, at least IMHO, was when the series stopped being a series about a ship and a crew. The splitting up the family is what killed it for me.
That's pretty much my opinion as well.
It makes sense for character to evolve and change, and a result of that would inevitably be having them leave the ship. Unfortunately, I felt that the stories started to get spread too thin with too many storylines happening in too many different places. I didn't necessarily mind the introduction of a second ship, but the time jump when everybody was suddenly spread all over the Quadrant just made it hard to care anymore.
Through the telling of the stories, crews of the Star Trek ships become families. When you split the family up, it ruins the dynamic of the story you're telling.