Oh, the time gem from "now" has gone somewhere, but it we know where it is in the past and we have a list of people who have it, and might think to use it creatively in the future which is actually already now our present.
Xavier is in the past, wondering where the hell his students are.
Time is broken to such a degree that it is possible to support a paradox like that somehow.
If his students never return, it's only going to take a couple weeks till he tracks them to the future, probably using Doctor Dooms Time platform, and comes for them.
Bendis faked us out.
But it could be a clue for what is to come too.