Remember that just then Khan managed to beam up the Genesis Device. This was officially Not a Good Thing.
Kirk is desperate to get it back so he then does the "you're going to have to come down here" line (which doesn't make any sense as Khan could beam Kirk up too). And Khan doesn't play along. So yeah, Kirk is not happy.
But he's also playing along at the same time. I think.