Where would someone get all of the raw materials to build such a structure? If it has the surface area of that many worlds….it seems massively unrealistic that something like this could be built.
I think you may be overestimating the requirements here.
Sure, the amount of raw material is huge... BUT, we're not talking about difficult to produce materials which require huge reosurces.
A Dyson Swarm or even a Sphere would have to be made from superior synthetic materials which can be made in sustainable abundance.
Now, better materials would have incredibly high strength, flexibility, etc... carbon based composites might do the trick... such as combining carbon nanotubes with synthetic diamonds and graphene (for example).
Mining Mercury with self-replicating automated bots would already provide more than enough material/resources to construct a full fledged Dyson Swarm (which would be made at exponential speeds) ... and it was projected it would have taken 50 years to do so... but, actually, much less time than that would be needed when you think about the premise the self-replicating bots would also be researching new materials and science, self-upgrade, etc... so newer sections of the swarm would be comprised of much better materials which take less time and resources to make.
And because science and technology evolve exponentially, building a Dyson Swarm would no longer take 50 years, but probably less than 25 years... I'd say closer to 15 years perhaps because the method by which you construct the said swarm would be improving all the time.
Already we are using adaptive algorithms to do R&D of new materials... and to look for new drugs (which is woefully underused).
Automation is key here.
So when it comes to an actual Sphere I think the whole Solar system would need to be dismantled along with the asteroid belt and Oort Cloud (however, a Sphere remains impractical atm because of impacts of gravity and other things that could mess up the structure).
A Swarm we can make with just Mercury (much easier and less destructive while providing pretty much most of the benefits a Sphere would provide).