You're certainly correct, but the Borg were something special. My solution to what you're talking about would probably have been to use them more sparingly, and introduce a new threat to use in those other stories.
But also, Locutus and Hugh did not diminish the menace, nor did Seven, really, if her origin had not also had to involve the stupid queen. The only things I would really say diminished them were the existence of the queen and the way Voyager overused the heck out of her and the Borg in general. And even the queen could have worked if she had been an anomaly that was managing to control part of the Collective, a part that would have returned to normal function if she was gotten rid of somehow. She just didn't work as "The Borg". She seemed more like a Batman villain.