New haven is lovely with plenty of culture and arts and history, and great food. I can literally give you a litany of places to go and things to do in New haven! It's very fun. I would say Fairfield is also a nice place to visit with a walkable downtown and great restaurants if you don't want to venture too far from NYC.