For me - and this is not necessarily supported by in-universe story reasoning - the fundamental justification for the Prime Directive is simple: it saves humanity from having to continually take responsibility for others. In short, it stops us from feeling obliged to go around the galaxy playing the role of interstellar policeman and charity. The cost & resource (and moral) implications of such a role are potentially staggering.
The stuff about allowing species to develop "naturally" is a polite rationalisation for this, but not, I think, the ultimate reasoning for its value. Without it, there is no consistent protocol for deciding what situations the Federation gets involved with. Given the potential dangers posed (not just moral, but practical) and the latitude Starfleet Captains already have, the PD is a simple way of ensuring avoiding contact unless you can be fairly confident of dealing with a species on an approximate par with yourself. Warp drive is a fair marker for this, because it means a species has both reached a certain technological sophistication and has a desire to make contact with the outside universe.
As I understand it, the PD applies only to Starfleet/UFP officials. Private citizens can do as they please, from what I can see. That also makes sense, I think, as we're talking about what the government wants to take responsibility for. Are there any example of private Fed citizens taken to task for breaking the PD?