>I think the worst possible idea is allowing a single person or handful of people to have near-total control over the future of AI

I'm not sure regulation is the biggest threat to the field of AI being open. We already live in a world where a small handful of people (i.e. decision makers at Alphabet, OpenAI, etc.) have an outsized influence on the development of the field because training large models is so capital-intensive that very few organizations can really compete with them (researches at universities sure as hell can't). Neither compute (on the scale necessary to train a state-of-the-art model) or well-curated large training datasets are cheap.

Since it is in the business interest of incumbents in this space to minimize competition (nobody likes to be disrupted), and since incumbents in this space already have an outsized influence, some degree of regulation to keep them in check may well be beneficial rather than detrimental to the development of AI and derived technologies and their integration into wider society (at least I believe so, although I'm open to other perspectives in this matter).

I'm not sure reading LessWrong will necessaryly disuade someone who is already a bit sceptical of the Rationalist EA community from believing that there is something culty going on. One of the things that really rubbed me the wrong way about that blog back in the day (I'll be up front and say that I haven't been keeping up with it for the past 10 years) was exactly how insular a lot of the writing was and how little it seriously engaged with existing literature and research in favor of reinventing the wheel and relying on their own private language which was not used by anyone else working in similar fields (as an example, Yudkovski is far from the first person to promote naive Bayesianism (basically the idea that if you get good enough at applying Bayes' rule, you will have solved the problem of induction), but if you only read his blog back then you could easily come to believe that he was doing groundbreaking stuff with respect to this topic when this was far from the case).