Murray's Cheese, a landmark gourmet shop on Bleecker Street since 1940, recently finished a large expansion and it looks great. I always try to buy cheese here-it's in my neighborhood and they are super friendly, knowledgeable and give me new ideas for things that I "need." They're obviously known for their huge selection of cheese but now carry coffee, chocolate and too many T-shirts and gifts. I do like their Big Cheese T-shirts though!
You can tour their famous cheese caves beneath the store in the "Mystery of the Caves" class offered a few times a month. And they have Mozzarella Making, Cheese 101, The Harmony of Wine & Cheese...everything uppity New Yorkers love to learn about.
If you don't live in New York, Murray's recently began to offer some of their cheeses in Kroger stores in Ohio, Georgia, Alabama, Tennessee, Kentucky and Texas (and in King Soopers stores in Colorado).