The only thing that seems rather silly is that whirring-clacking sound when the computer was working, which was of course deliberately weird or cute or something, since I don't believe anyone back then thought the Enterprise's computers were actually mechanical either.
Don't be so sure. "The Cage" showed the computer producing a paper printout of its findings. And the data storage devices were repeatedly referred to as "tapes."
And even if it was dramatic license, it wasn't to be "weird" or "cute," but to be accessible to an audience that had certain preconceptions and expectations about what constituted a machine. Just as modern Trek shows unrealistically depict visible phaser beams in space and show ships implausibly close together, not to be weird or cute, but to present the scene in a way that conveys meaning to an audience with certain assumptions or expectations.
Besides, it's not as if modern computers are totally silent. You can hear your computer's hard drive making noise when the computer is opening or saving a file or performing some other memory-intensive task. It's quiet, but it's there. There are moving mechanical parts inside 2008 computers. So it's not unreasonable that people in the 1960s would've believed that computers centuries in the future would have mechanical relays of some sort.