The Nerdist sounds like it'd be right up your alley. I particularly liked the interviews with Neil Gaiman, Neil deGrasse Tyson and Danica McKellar--those were all fantastic podcasts IMO.
Ahh that's the one I meant, not Comedy Nerds.
I gave up on that, but can't entirely remember why!
They can be really hit or miss, and I generally focus on the ones with guests I'm interested in (and sometimes those guests are boring as sin too).