If there's a store like it in DC proper I definitely haven't found it. You can find a lot of the component parts (Big Planet Comics is right down the block from Joint Custody Records which combines to be like 50% of what a Newbury Comics would have) but there isn't something that perfectly nails it.

I'm not entirely sure if the market is there. DC probably has the least room for quirky fun stuff of any major US city. People take themselves too seriously.