I've not found refrigerating bread to make it go stale, if it's sealed up well with minimal air inside the plastic bag. It'll delay it from developing mold, and I have plenty of experience in that. But yes, cooler air can make moisture wick out and make it dry. That's why I try to buy a whole loaf and slice it when I need it.
As for freezing, moisture wicking is more of a problem, so you don't want to leave the bread in there for too long (depending upon how much air you've evacuated from the bag). But I've found that softening it up a little in the microwave prior to toasting tends to achieve a more natural consistency than toasting it frozen.