. . .it doesn't make sense to me that he got promoted to the rank of Captain and then got bumped back down to Commander. . . So why couldn't Riker have retained the rank of Captain but still been first officer?
Again, it was a field promotion.
"Commander Riker, I hereby promote you to the field commission of Captain. The
your ship now. Congratulations."
-- Admiral Hanson
Hanson died during the Battle of Wolf 359. It's possible official orders were never drawn up. It's also possible Riker personally requested a reduction of rank after learning Picard would return to the Enterprise.
For further information, see Memory Alpha's article on provisional officers, which specifically mentions Riker's situation. http://en.memory-alpha.org/wiki/Provisional_officer