Babylon 5 is much closer to Trek and shares many elements with DS9 (

): a space station, interstellar wars and a healthy dose of character development. It's not a utopian future, humanity as a whole are not virtuous, but the human cast are all pretty much good people while the alien cast evolve over time. It's not quite as polished as Trek, but the show has a lot of gusto and the plot holds together very well as it was all planned out in advance. Season 1 isn't very good, there's plenty of bad episodes in there and it's very cheesy, but by the end of season 2 the show had improved a lot.
BSG is very different. It's a bleak show, which it should be because it's about the genocide of the human race. There's no aliens, there's just humans and Cylons (with hints of an external force that I can't tell you about because it would be a spoiler). The show is almost as far away from the utopian vision of humanity as you can get, it boils down to humanity trying to survive on overcrowded ships while Cylons try to blow them up at every opportunity. All the characters have flaws and they struggle to overcome those flaws in order to survive and make it to the mythical planet of Earth. Contrary to B5, the show is extremely polished and looks fantastic, but the story arcs weren't planned out and in season 3 it becomes a big mess that's difficult to justify. Also contrary to B5, the first season and a half are the show's strongest (although there's still much to enjoy in season 3 and 4).
They're both very good shows with unique advantages and disadvantages. BSG had three staff writers from DS9 working on it (and one from Voyager), but B5 is closer to DS9 in setting and tone.
(And if you do decide to give BSG a go there is one important thing you should know:
The Cylons did not have a plan. That tagline was a gimmick to hook viewers, the show makes much more sense if you accept that the Cylons are just winging it from the start.)