I do think sometimes we attribute our feelings (as fans) about the ships to those of the characters. Whereas we
have an affection for the 'TOS asthetic' and find it hard to think the characters would be able to relate the sterile grey of the refit to the original on those terms, to the characters themselves it might seem like there is little difference between the floor-plans of the two versions of the ship. To Kirk, stepping aboard in that cargo bay, the ship had a very different look, but was still exactly as he
remembered her. In this sense you could
almost argue the case of it being a retcon of sorts.
I also think Mad Magazine were very clever in their parody of the Enterprise refit at the time. The bit in the movie where he's told "Admiral, this an almost entirely new Enterprise!" became a hilarious scene where Kirk tries to go to the bridge and instead ends up in the men's room.