The original idea of having it be Wookiees would have been much better.
This is no doubt true in terms of making the film more believable relative to combat mechanics, and I used to believe the film would have been improved by using Wookiees instead of Ewoks.
I changed my mind, though.
I believe that the key point is that there's a limit to how intense this film can be and still have the OT conclude on a high note. A thoroughly intense scenario would have had the jungle burned down for some distance all the way around the Imperial facilities on the Forest Moon. Sneaking up in the bushes would have been out of the question for any species. After an incredibly brutal fight, hitting all the parallels with Vietnam, including Wookiees on fire, there's just no way that the audience would be anything but drained. If you want it plausible, children in the theater are going to be crying.
As already mentioned, the Jabba's palace chapter and the Emperor throne room scenes have their measure of intensity. After TESB, the trilogy needed to lighten up, and get back to the pure fun that the first film brought to the screen. I don't believe that ROTJ succeeded in every way that EPIV did in that respect, but I believe that that was the intention.
In particular, the Ewoks don't entirely succeed in execution. For one thing, the Ewok suits that were used to make the film were pretty poor by 1983 standards, but overall they seemed to make the best out of what they had.
It's also worth pointing out that the idea that Wookiees would be underestimated is much more far-fetched than the idea that Ewoks would be overlooked. In a realistic scenario, Imperial troops would have drilled specially for Wookiee attacks, were Wookiees indigenous. Ewoks on the other hand it seems were simply thought to be stone age aboriginals and were wrongly judged not to be a threat of any kind. So, the Ewoks enjoyed one tangible advantage over the Wookiees, in terms of surprise and lack of specific Imperial preparation.
at least the Gungan army in TPM was only there to provide a distraction, as they lost to the droids after a fairly short time. Somehow the Ewoks manage to defeat "an entire legion of [the emperor's] best troops" with ease.
And this is certainly so. But then again, there's not as much at stake at that point in TPM, only one planet. In ROTJ, it's one do-or-die battle for the fate of the whole galaxy. Were the stakes and especially the objectives different, the Gungans would probably have fought a different kind of battle, besides stand-and-defend with the intention of simply falling back when they couldn't hold the position any more; their battle plan didn't require them to defeat the army or occupy a military installation. In ROTJ, they
had to do those things.