Notice how it's being called 'iced tea'? That's because people say 'tea' for hot tea, then 'iced tea' for cold.
In your experience, yes. In the southern USA, asking for tea will generally be interpreted as requesting something ice cold and syrupy sweet. You would need to ask for hot tea, to differentiate.