There are a lot of great lakes with swimming areas all over the state. These are better suited for babies than swimming holes. Swimming holes usually have large rocks and deep sections.

Here is a nice quiet spot: https://vtstateparks.com/plymouth.html