Fire said:
Because Starfleet design ships specifically for their tasks
The classification system we got in the Technical Manual is really very general, and the term "starship" carries the general connotation of a large, multipurpose vessel. In TOS it seemed to be, in fact, a rather specific technical term. Starfleet ships of the line serve many duties: they explore, carry out scientific investigations, transport passengers and cargo, defend Federation members and allies, and so forth. Such a diverse mission profile, as well as warp physics, calls for the large, complex and heavily crewed starships we see.
I suspect the variety in design we've seen chiefly has to do with the fact that viewers encounter Federation starships more often than those of other powers. We really don't know if Romulans, Klingons and others have such a diversity of designs, and I'm sure the creators would have loved to give us a dozen Romulan warbird types instead of what we got and encourage us to have fun with such ideas.
In-universe rationalizations aren't too tough; considering the great diversity of cultures and technology within the Federation, diversity in technology and aesthetics and frequent updating in the state of the art are probably the norm. Remember, though, that several designs of Federation starship have evidently remained in frontline use for a century or more, whereas some newer designs are only seen once or twice and may represent unsuccessful paths of development (example:
Sovereign-class starship). Perhaps many small runs of new, different types of starship are "tried out" until highly successful classes are hit upon, representatives of the class then being built in much greater numbers (such as the
Excelsior class). This seems a sensible approach as long as the technology has a lot of modularity at the "nuts and bolts" level.