The army found it though, and I guess they would have had a way off.
Maybe. Although, the Others killed all the army guys. We didn't hear about any of them leaving. Which isn't to say they didn't of course. But maybe the army somehow found the island and got more than they bargained for when they got there.