I've lived in a number of cities large NYC, London and small like Manchester, and others you haven't heard of.

By and large, with regard to safely, how one carries themself when out and about in any city matters more than the city and neighborhood they're in. There is no where in Manchester I don't go due to safety concerns.

NH is sufficiently affluent and suburban that a small percentage of the those venturing into Manchester are horrified to see poor people, sometimes living in the street. And they'll lose sleep over being asked for change or a five. Its mostly these folks who give Manchester a bad name and scare other people.

Also, know that almost 10% of the population of NH lives in Manchester, so it's quite different from most of the state: many more people, density, homes, etc etc etc. More. More is often bad to suburbanites.