Earl Grey only refers to tea, and tea is almost always served hot.
Every time I went to work in the USA it took the first week of my hotel stay to explain that I didn't want iced tea with breakfast. From then on the server would remember me as the funny British guy who wanted 'hot tea'.