It's also very possible I'm just unaware of those spots, the intent of my visits was never to go out drinking. I'll have to go and see for myself next time I'm there!

Raritan Valley line does require you to transfer in Newark Penn most of the time which usually is not a big deal. u/New_Stats is correct that the last trains back to Newark from NYC typically are around midnight or 1. You could potentially take the PATH train back to Hoboken/Jersey City for an Uber/taxi that doesn't have the NYC price tag on it if you're out of options.

Also funny enough I'm familiar with New Hope (Lambertville not so much) too, they are cool towns with great restaurants but it's not the kind of nightlife where you'll be out at a bar with loud music and a DJ until 2am, at least what I've seen. Not sure what your desired 'nightlife' situation is.

I don't live there but have visited frequently over the past few years.