First...I agree. Metro North is way to go.

But...for the others...My advice with Amtrak is it is only good if you are coming from Hartford or New Haven in the morning only. Hartford line does not connect well with metro North. It's super annoying

If you buy in advance (by at least a week) the ticket has been 20 to 30 bucks for the direct one in the morning that arrives before 10am. The cost is only a few bucks more than peak train and way more comfortable especially in early morning. The afternoon trains for amtrak are a mess always... especially northbound. Delays of hours are super common. Not worth the risk just hop on the mtn.

Edit to clarify the physical connection is easy but there is often a wait time of 20 to 50 minutes. Only reason i mention but still good option considering parking in New Haven is expensive