I think there are two basic problems with Voyager. The first is the poor development or lack of development of the show's original premise. This is a valid criticism and one that I share, but it is not why I didn't like the show.
I like Star Trek. Even mediocre Star Trek is something I enjoy, though I can recognize that it is mediocre. That is basically my feelings toward Enterprise, which I'm currently watching for the first time on DVD. Voyager is the only Trek show that I have never been able to get into and enjoy, despite the occasional good episode.
Poor development of the premise is not sufficient reason. I realized at some point that it is the crew. They are such a boring, uninspiring group. Even with Enterprise, I find I like the group of people I am following on a weekly basis, and I like their group dynamic. It is not DS9 or TNG, but the crew basically works, and I find the mix of personalities fun and enjoyable. With Voyager, I just can't get over how bland and insipid the crew is, both individually and collectively. This has to be partly the writing and partly the acting. The chemistry, the charisma, the group dynamic, whatever it is that makes the other Trek crews work, the Voyager crew just doesn't have it imo.
I find this frustrating, because in theory I'd love to have seven more seasons of Trek to enjoy.