I subscribe to a handful of podcasts, and the one I most look forward to downloading every week is NPR's All Songs Considered. Sometimes they play tracks from upcoming albums, sometimes they dedicate the whole hour to a live performance by one band, and sometimes they ask a group of music critics to comment on their favorite new releases.
But the best is when they invite musicians into the studio to be guest DJ's - to have a conversation with host Bob Boilen and talk about where their influences come from and what music has had an effect on their lives. I think hearing artists talk about what inspires them might be one of my new favorite things (which would explain why I have recently started going to a ton of artist talks at local museums and art schools.)
Some of my favorite guest DJ hours since I started listening to the show have been with Bright Eyes, Joanna Newsom, Weird Al and M. Ward. All are available in the online archive and are definitely worth a listen.