The last three panhandlers I gave to I regretted. People told me they were well known and had been working that corner/street for years. I just get a little soppy sometimes but you know there are charities for that.
I do give to buskers all the time. We have a ton of buskers here and an increasing amount of panhandlers.