You can find a 2bd/1ba <$3,500 easily in JC Heights. Would suggest living near the 9th St light rail or near Riverview Park. It's also possible in Downtown but in an older (but nice) building. All the new construction in DT, JSQ, and the waterfront in JC is fairly expensive. Hoboken is all around expensive. The Heights isn't as well-connected as Downtown but it's got it's own vibe and plenty to do, and you can always take the light rail around Hoboken/JC/Weehawken or transfer to the PATH to go into Downtown/Midtown Manhattan.

In terms of safety, JC Heights is very family oriented. Obvs keep your wits about you, don't be careless in any city. But I've had no issues and often see young families walking at 8pm/9pm and people walking their dogs until 10pm. I've had no issue on the light rail and walking from 9th St to Riverview.