I think that by the third season, Abrams had already left the show to begin development of Lost.
He was also developing MI:3, if I remember correctly.
Season 4 suffered from lack of direction and ABC shuffling around the episodes which consequently made some things make little chronological sense. I always watch season 4 in production order
makes it a little more bearable.