Peter's convinced he has to get back to his timeline but September only talked about him being erased from the timeline and then returning. I think he's going to realise at some point that there is no timeline to return to kind of like Planet of the Apes or something.
That would be a bad move by the writers especially if there is any season 5 or to hold viewership at current levels. The fans will feel like they have been robbed of the characters they loved from season 1-3 if the prime timeline is gone for good.