There are tons of good recommendations in this thread. Heading north, we really liked Portsmouth New Hampshire. Heading east, we like the North Fork of Long Island or Old Saybrook CT. The Poconos are fun, but you could also do Lambertsville/New Hope and kayak in the Delaware River.

Going south I'll throw out something I haven't seen yet - take the Cape May-Lewes ferry to Delaware and check out Chincoteague on Virginia's eastern shore. Very relaxing, and you're not going to find wild ponies anywhere else on the east coast. As long as you do it before Memorial Day you should be OK.