Sake of argument of a good handful of comparative pound for pound examples and associated tradeoffs, Jersey City can be cheaper with things but it's not that much cheaper than many situations in NYC. It doesn't take much to find some extremely lateral setups in Jersey City and parts of NYC.

If you're in a higher tax bracket where the squeeze of the tax man is a bit more noticeable with what you're taking home, than yeah "it's cheaper" but it's not like New Jersey is known for being some magical wonderland people seek out for low taxation.

If a concern is quiet and you have someone going to work in Financial District, then yeah I wouldn't say somewhere like Downtown Jersey City is that bad of ideas.

If there's any stuff I'd say be mindful of, if you have a friend group that is scattered all throughout NYC and you're in JC expect often being the one to be going out to meet up with people and factor in how the PATH can be a total clusterfuck during late hours and weekends. If you say money isn't really a concern, yeah knock yourself out with the more expensive uber rides home if you can find someone willingly to go into NJ but if you lead an active enough social life taking you over creation in NYC, it can get a little old and pricy after awhile.