According to Kirk in "Metamorphosis", the translator reads the brain wave patterns of the subject and matches them to known universal ideas known to be common to all forms of life.
In later years, it was established that this, in combination with the basic linguacode translation matrix, allows basic communication, with the translation becoming more and more accurate as a translation matrix for the specific language is further defined by continued use. After a while, it doesn't need to read brain wave patterns, as an adequate translation matrix is then in use for the particular species.
So essentially, with new species, the UT reads brain wave patterns, matches them up with linguacode to bootstrap basic conversation ability, and then learns as it goes, until eventually the thing is translating quickly and on the fly.
This scene is STILL stupid though:
Hoshi: Something called the Rom-illan Star Empire.
T'Pol: It's "Romulan"
In this scene, T'Pol is basically telling Hoshi that she is incorrectly pronouncing a brand new English word invented by Hoshi's translator to represent the Romulan word for their species.