No, I don't think the Davos actor would work as Stannis. Remember that Robert was the eldest brother. Not Stannis. Stannis is not as old as one tends to imagine with the unflattering descriptions in the book.
Liam Cunningham and Mark Addy are only two years apart in age, so Cunningham playing a slightly younger brother (although he's two years older than Addy in real life) wouldn't have been a problem. Anyway, Stephen Dillane, who is playing Stannis, is actually seven years older than Addy and five years older than Cunningham.