My general impression is that during the 1950s/1960s Latinos were for the most part seen as white. Montalban certainly was, since both of his parents immigrated directly from Spain, meaning he almost certainly didn't have any indigenous blood.
Certainly from what I've seen of popular media at the time, it was considered more scandalous for a Jewish guy to date a "white" woman than a (white-looking) Latino guy.
Although apparently Lucille Ball had to fight to be married to Desi on "I Love Lucy." From Wiki:
"In 1950, CBS asked Ball to take
My Favorite Husband to television with co-star
Richard Denning.
[27] She, however, saw a television show as a great opportunity to work with Desi, so Lucy insisted that Desi play her husband, much to the dismay of CBS, which was reluctant because Arnaz was Cuban.
[28] CBS executives believed that audiences would not believe the marriage between an all-American girl and a Latin man.
[28] To prove CBS wrong, the couple developed a
vaudeville act, written by Carroll and Pugh, that they performed at the Newburgh's historic Ritz Theater with Arnaz's orchestra.
[29] The act was a hit and convinced CBS executive
Harry Ackerman that a Ball-Arnaz pairing would be a worthwhile venture. At the same time, rival networks
NBC,
ABC, and
DuMont were showing interest in a Ball-Arnaz series, which Ackerman used to convince CBS to sign the duo."