Not really adding any new suggestions that haven't already been said but just reiterating really how many options you have based on what you're looking for.

Guilford and Madison are both great choices and would put you about 20-maybe 30 minutes from Yale. They're both kind of "long" towns so closer to the shoreline will be closer to their downtown and North Madison/Guilford is a bit more wooded. Both have great school systems. Closest Whole Foods and Trader Joes would be Milford but there's also a Fresh Market in Guilford, plenty of other shops and restaurants in both towns, and about 15 minutes (give or take depending on which town you end up in) from Branford in one direction and Old Saybrook in the other which both have Walmarts, Kohls, etc. Both also have train stations but to get to NYC or Boston you'll generally have to go to New Haven or Old Saybrook (there is a connecting train to either often driving to the other station is easier).

Branford and Milford are also great. Both are a bit more commercialized than Guilford or Madison but also have areas that are just as quaint and "New Englandy" if that's something you want.

Westbrook, Old Saybrook, Essex, Old Lyme area also has a lot of what you're looking for though you are getting farther away from Yale at that point. These tend to be smaller towns than the first four I mentioned. The Waterford/New London area has a lot of chains so even though you're farther from the New Haven area you'll still have access to many of the same stores/restaurants.

Having grown up on the shoreline I know less about these towns but as some others have mentioned: Trumbull, Orange, Woodbridge, North Haven, and Cheshire are all worth looking into as well.