I think that in the Avatar movie world things are not so simple as that. From what I gathered it was a very secret operation on just how they were getting that ore and if the public knew it would be over for them. Nuking a planet would definitely get more notice than an attempt to quietly move/subdue the indingenious people.