Data must look particularly artificial in person, more so than on the TV screen. There must be something about him that screams "artificial" to an individual from a more technologically advanced society. When he was stuck in the 19th century, no one seemed to think Data was not a human, just that he was particularly pale. Clemonds, though, immediately picks up on Data being something other than human. I wonder if 24th century individuals would initially have an even stronger reaction to Data than Clemonds did.
I would assume that Data has a disconcerting air about him. He probably lacks a natural scent or body language, which could be a bit off putting.