"The Federation is too small compared to the Galactic Empire."
That's quite possibly the answer to the question of why Trek ships are puny compared to Star Destroyers. Are we talking pure combat here? Then consider: The Federation is a small galactic power. The Empire spanned the Star Wars galaxy and has access to a galactic level economy and resources. This is why Imperial Star Destroyers are even considered "small" capital ships in their fictional setting.
Look at "large" galactic powers in the Trek universe: The Borg and the Dominion. The Borg have no economy to speak of, they just hoover up resources and build massive cubes to continue the process. Their ship sizes make sense for that purpose. Each cube is very probably a planetary assimilation factory. The fact that it's also a starship that can attack other ships is its secondary role.
The Dominion actually have a ship that's roughly equal to the size of a Star Destroyer. I'm sure everyone has seen this:
Dominion scaling
(I'm also assuming the DS9 finale with the 5km battleships was a scaling error and that the battleship is 1.5km long)
The Dominion fielded four or five of these battleships at the most as seen on screen during the Dominion war. Guess what they fielded lots of; the teeny tiny attack ship in the top right-hand corner. The ship that's cheap to mass-produce, can be fielded in large numbers and can take down a fleet of bigger ships like a swarm of angry wasps. In a setting where smaller ships can be extremely effective in combat, a bigger ship is just a bigger target.
The Dominion is probably large enough with enough resources and an economy that can support a large number of big scary battleships, but they still build thousands of small attack ships that pack a punch, are hard to hit, and are easily replaced. Dominion battle strategy is also quite effective: They send swarms of their smallest attack ships to kill a fleet of bigger ships to deliver the message: "This is what we can do with our weakest ships. Imagine what we'll do if you have our full attention."
Remember this fleet?
Romulan/Cardassian fleet
20 ships. 4 Warbirds, 16 Cardassian cruisers. All got turned into swiss cheese by 150 of those tiny attack ships.
The Romulans are a special case: they try to field big impressive battleships in the 24th century. In fact, that's all they appear to field and they get creamed every single time they're seen on screen in combat. I don't know how many Warbirds the Romulans have at any given time, but it seems to be measured in the dozens, not hundreds.
As much as I loathe the comparison, I'm going to use a Wet Navy example. The Empire of Japan built two massive super battleships, 70,000 tons each, the biggest pure warships ever built with the biggest guns ever mounted on a battleship. Their purpose was to outclass anything the United States could build, because they couldn't compete in terms of sheer numbers of ships. Yamato and Musashi were big and slow and helpless against aircraft. The U.S with its bigger economy just built more ships.
For the purpose of war, you can only build and maintain what your economy and resources allow. Technology has a lot to do with it, but the general rule is: the bigger your nation, the more you can build and the bigger you can build.
Now if we're talking about peacetime, and exploration and diplomacy, then you don't need troops and fighters and ships packed to the brim with the biggest heftiest weapons (although a few are nice to have). You need ships designed for a specific mission profile. You need just enough crew to fill that profile, because crew consume resources in the same way warp engines use up antimatter. So you have to concern yourself with fuel requirements, on-board supplies including food (I choose to believe replicators turn organic goo into food, not convert energy which would be inefficient and stupid), regular maintenance and repairs carried out for the duration of a mission and when docked at a base, and so forth.
For all these factors, smaller is in fact, better. You should want the simplest design to fit a mission profile. You should want to expend the least amount of energy needed to complete a mission. You should want to risk the minimum number of lives. You should want to be conservative.
