Like I said, it's Gilligan Syndrome.
Other shows manage to avert it (Farscape being a good example) because they had OTHER plots beyond "Go Home". By the time Crichton got back to Earth the Scarran/Peacekeeper/Wormhole tech plotline had become more important.
If VOY had another major plot to drive the show beyond "Go to Earth" then there'd be nothing wrong getting home earlier as long as they could continue the other major plots (which would involve leaving Earth again).
But it didn't, which is a failing of the show that they relied on a limited plot.