It wouldn't work, for the simple reason that the styles of the two are so different. Star Trek characters (Starfleet anyway) are rather po-faced, whereas the Doctor comes across as a carefree mischief-maker.
Basically the Trek characters would treat him like Q, Trelane, The Outrageous Okona, or Harry Mudd, and they would be like any number of the various blustering space ship crews the Doctor has met over the years..
At first they would - and then he'd team up with them and save the day and they'd realise they're wrong.
The whole point of DW is that you can put him into *any* style of setup or series - cop show, western, period drama, horror, reality show... And he'll still fit.
The only people who think "oh, they won't fit" are really fans of one series who look down on the other.
Now, they wouldn't work as a regular thing, or a series: Star Trek Gallifrey wouldn't work, cos Trek is all about (Western) humans pulling together for truth, justice and the American Way, while DW is about individual libery and responsibility - but on a single, individual basis, a visit from the Doctor would pretty much equate to an episode where one of the crew discovers that they *can* do what it takes because they're an individual.