ISTR reading that even as late as 1991, one of the biggest sticking points with Harve Bennet's plan to do 'Starfleet Academy' was in part that while TNG had proved that a new
crew could actually work, there was still a lot of uncertainty among the Paramount execs about the idea of recasting the characters Kirk, Spock and McCoy, and whether that would taint the TOS part of the franchise. With the benefit of hindsight we can see that it would have been workable, but I think it's really only been in the current climate of reboot movies that people have become much more accepting of the concept that iconic roles like that can be reinterpreted and recast. I certainly don't think the idea would have been welcomed a decade earlier. At least Phase II had the good sense to make Xon a new character.
As far as the topic goes, I don't think it would have worked in context of TSFS either. The whole point of that movie was to give Leonard Nimoy an "in" to allow him back into the franchise. Who knows how it'd have turned out if he did walk away from Star Trek? Do we think the producers would have gone for a Spock-less Star Trek? Hard to say how that'd have turned out.