Them not being all one big happy family isn't the problem. The problem is the fact that aside from Rush, none of the supposed leadership (Young or Wray) seems to be able to figure out that they should focus on increasing the odds of their survival instead of wasting time and energy on petty squabbles that decreased their ability to survive in the long term.
But... why do you think there are these "squabbles" are happening in the first place? It's because they're all desperate to survive, but disagree on how to best accomplish that. What sane adult would hush up and do what their leader told them to even if they thought he was inept and liable to get them killed? The military are trained to, sure. But the civilians? Fuck that.
Other than Dr. Rush, no one seemed to really understand what was going on. I could understand if they were arguing over competing plans over the best way to get home. But no one besides Dr. Rush even seemed to have the foggiest idea of where to start on that front. Mostly, they were just bickering because that's what so much of sci-fi is nowadays: People bickering like petty jerks whenever the going gets tough.
I'm wondering if it had started out a little more exciting with aliens and stuff actually happening if the ratings would have been sufficient? Or was there some Stargate fatigue going on?
Hard to say. IIRC, the ratings for
Stargate Atlantis were on the decline towards the end, which is why the Sci-Fi Channel ended the show. So, the
Stargate franchise was not on the strongest footing to begin with. But then
SGU just killed it by being slow & depressing & nothing at all like the 2 previous shows. And the people who might have been interested in a dark, broody show like
SGU didn't give it a chance because it was related to cheesy popcorn shows like
SG-1 &
Atlantis.
My personal theory is that
SGU would have been more successful if either:
1. It had been the 1st new spin-off, not the 2nd. When
Atlantis began, I think fans would have been open to the spin-off taking a significantly darker tone than
SG-1. But when
Atlantis demonstrated that it was just going to be the same show with different characters, that created a pattern. Once that pattern was established, I don't think too many fans were interested in a 2nd spin-off that radically reinvented the franchise that most fans thought wasn't broke to begin with.
OR
2.
SGU had aired concurrently with the 6th season of
Atlantis. I think fans might have been open to a much darker incarnation of the
Stargate franchise if it wasn't the ONLY kind of
Stargate they were getting at the time. If they'd set up
Atlantis as the light flavor &
SGU as the dark flavor and put them next to each other, I think people might have been intrigued enough to give it a try. But when
Atlantis got cancelled and
SGU became the only
'Gate in town, I think most fans felt safer jumping ship altogether and basking in their warm memories (and DVDs) of
SG-1 &
Atlantis rather than try to muddle along with this new abomination.