True enough, but nothing OP indicated that.

The property owner isn’t kind of or otherwise screwing you. Sure, it isn’t the situation you wanted. But you will be fine.

It’s pure supply and demand. The people who live in these towns do not want more housing build, especially anything a younger person and family can afford. So the zoning rules and permitting process is so difficult and costly that it doesn’t happen.

Why don’t they want young families? My tax bill is ~64% spent on schools. Every little cherub costs almost $20,000 per year. Much more if they have some special needs. A family with 3 kids will cost almost 3/4 million in current dollars through high school. That household will pay less than 10% of the costs with their tax bill and of course they also get roads, police, fire, dump / trash etc.

Towns are way better off attracting older empty nesters or a massive airbnb population who uses a lot less for the same tax payment.