Now imagine similar tech applied to smaller StarFighter class sized vessels.
OK. Cool. I love your idea, though I'm more of Gundam meister myself.
But, I'll counter with an equal but opposite thought...
Let's take your starfighter, and now imagine similar tech applied to larger starship class-sized vessels. There is no reason why a large starship cannot have the same technology, except that you wouldn’t want it because you would want superior technology! More power, more speed, etc. Granted, in Star Trek, we see that capital ships have mostly traded some tactial advantages for more toilets, more comfort, more cargo, etc. It seems unlikely, though, that they would have done so to such an extent so as to make the capital ship unnecessarily vulnerable to fighter and drone attack.
As a countermeasure—besides simply warping away before the fight—to a swarm of surrounding vessels creating a large spherical envelopment by maneuvering almost randomly like crazy, anitmatter spreads should be used to create a flak canopy, which will reduce the volume of space where fighters and drones want to be. In addition phasers and photons should fire rapidly and use proximity charges in the now more concentrated volume of space where fighters and drones have been forced to operate in.
Any small fighters or drones vector-thrusting almost randomly like crazy might just as well run right into ordinance rather than avoid it, because if we believe a fighter is faster than a capital ship, we have to believe that explosive ordinance is faster even than a fighter.
All of this supposes, though, that the small weapons used by fighters would even affect the hull and shields of a much larger and vastly more powerful vessel at all. It might just be that their maximum firepower is simply not enough to penetrate hulls and shields of larger more powerful vessels with massive power generation and seemingly limitless power, even when added up all together. And, of course, capital ships would be engineered and constructed exactly that way because they could be! In Star Trek maybe they have been.
To damage hulls and shields, fighters would need to bring to the fight ordinance capable of doing so, which brings us back to the photon torpedo. This changes the role of the fighter completely. It seems that a starfighter’s primary purpose would be to counter defenseless targets and other starfighters, not capital ships. Secondarily, to counter capital ships, a starfighter must attempt to deliver its ordinance (or itself if a suicide attack) before it is lost and then resume its primary purpose, if possible.
So, it becomes a battle of attrition where the wisest use of resources on an empire level may not win every battle but ultimately wins the war. In a war of attrition, the photon torpedo reigns supreme because you do not have to send people with it, you can afford its expendability, you do not have to teach it, you do not have to retrieve it, and you do not mourn when you lose it.