Downtown Providence:

Cellar Stories for used books, magazines, ephemera. Could spend hours in here.

Symposium for new and used.

East Side:

Books on the Square is good for new. I think they also have some used books.

Paper Nautilus for used books.

Twenty Stories for new contemporary books. Get a book then go next door to Aleppo Sweets.