Vernon Hill resident of 4 years or so here, also a woman. You'll be fine. The worst to be said of Vernon Hill is it can be noisy sometimes because of the population density, but I've never felt unsafe and I walk around a fair amount. Also, there are lots of non-white people, which some people find scary.