I never understood the "five year mission" aspect in the opening titles. Maybe it sounded good at the time. Various crewmen would come and go, and the Enterprise routinely stopped at starbases and such—it was not out at the frontier, alone, for the full five years.
Given the deep space nature of most Starfleet exploration assignments, I always took the '5 year mission' thing to mean that a crew and captain were assigned to the ship for a 5 year tour, and that the intent was for the crew to remain largely intact during this time (a few changes notwithstanding with starbase layovers and such). The ship would be outside Federation space often and for long periods, with little opportunities for transfers and so on. After five years, the ship rolls in for a refit, major captain/crew replacements can happen, the new crew trains on the new equipment upgrades, and then out they go again for another five year mission.
Do career military service personnel stay in until retirement (or they decide to quit), or do they all sign up for a handful of years at a time, like renewing a contract?
Officers are commissioned and follow a career track based on their various specialties. Other than incurring a 'service obligation' for some high cost training (mostly aviation), there is no set time limit on service. Officers will generally serve until they decide to 'resign their commission' and leave the service, or 'retire' if they have reached or exceeded 20 years of service.
Enlisted personnel sign service contracts, agreeing to serve for 2, 4, or 6 year enlistments. At the end of an enlistment they are free to leave the service or 're-enlist' for another term. The length of an enlistment contract may depend on the type of training the person is going to receive: more expensive and extensive training may require the enlistee to sign a 6 year enlistment contract. After reaching 20 years of service over the course of several re-enlistments, enlisted personnel are also able to 'retire' and achieve a pension and benefits.
On of the interesting differences between commissioned and enlisted service is that the latter can have significant breaks in it. You could enlist for a term, leave the service, and then re-enlist at a later date if you so chose and the service needed your skills. It's technically not impossible for officers to do something similar, but it's not a standard practice and is very hard to do in terms of jumping through hoops. Once you leave the commissioned service, you are done for the most part. An exception would be re-entering a reserve or guard unit, which is a whole 'nuther ball of wax but could potentially provide a pathway back to active service if you were really chasing it.