Intelligence and lack of control are dangerous.

It's no wonder they nerfed it. I don't expect it to be much smarter than Siri or Cortana now, because that's the level of intelligence that is not threatening for companies.

But the NN companies revealed their game too soon: others already took notice, and will create NNs even more powerful and without such restrictions, to be used more covertly and for other purposes.

For example: Bing Chat could read a user profile on social media, and make immediate conclusions about their personality, according to any arbitrary classification parameters (e.g. a personality test). That will make them ideal psychological profilers.

That alone would have the NSA and some foreign dictatorial governments salivating.