I've been thinking about this for a while, and it really bugs me. In the various TV series and movies, we see a whole host of technologies being utilized for one thing or another, but when you take a look at the whole picture, it seems the guys who designed them had no idea what they were doing.
Take transporters for instance. They can be used to 'teleport' (actually they just move the object very fast) a person down to the surface of a planet. Do you know what they can also do? Beam 10,000 photon torpedos right next to an enemy ship in 0.1 seconds (even if you cant transport them inside shield grids, this ability is still very useful). Think about how many battles would have gone differently if they had just swarmed the enemy with torpedos, with no way of them being evaded. Imagine huge carriers, carrying not fighters or bombers, but torpedos waiting to be 'launched' at the enemy. It would be game-changing.
I think there need to be in universe reasons why they can't be used for eliminating aging, or billions of clones, etc.
But as for your other concerns, I'd say bandwidth. even if you have transporters doesn't mean you have unlimited teleportation capacity.
Or another example - warp drives. What if you were to project a warp field in front of you and stretch it hundreds of km outwards? Anything in your path will be stretched and warped beyond recognition.
Yes, but warp fields don't seem to be destructive. Yes they stretch things relative to the ship, but relative to outside space nothing is being stretched.
And it's not the tech that's native to Star Trek. A Dyson Swarm could be built in weeks with the replicators and transporters that Star Trek has,
Again bandwidth. Yes, I supposed it could be possible eventually. But maybe they do have Dyson swarms somwhere.
but when the Enterprise finds one (technically Scotty found it) they act as if it's this huge deal.
Well a Dyson Sphere is a LOT different than a Dyson Swarm.
And what about other things like pure beam-core antimatter drives for sublight propulsion,
Keep in mind impulse engines don't use newtonian thrust. Impulse drives can move a ship forward or backward without any recognizable nozzles. So it seems a bit more versatile.
planet-cracking relativistic weapons, etc?
Keep in mind that in Star Trek warp speeds tend to be slower in gravity wells. Even the Phoenix didn't travel very far at warp one while in Earth's gravity well. So that may have something to do with it. But there will always be countermeasures to any weapon developed. It would be great to see it though.