Depending on the neighborhood you live in, you can get away with that. My childhood best friend had a cat named Jackie who, for all intents and purposes, was a stray. She had a litter box and food in their garage, which they always left cracked open, but she was constantly roaming the neighborhood. Lived to be about 20-years old, too!
I'm too paranoid about my cat to ever let go outside for anything. She's a little wieny and would probably die within 5 minutes.