I think he’s probably too far on the pessimist side, but we need people presenting both extremes. The truth is usually somewhere in the middle.

Maybe. We run on prehistoric instincts and drives which can lead to all kinds of bad things. Maybe augmentation will mean the rational side of ourselves wins out and we can improve things like willpower/self reflection/empathy.

A lot of wishful thinking with little evidence to back any of this up.

You assume that absolute power will not corrupt these philanthropists. You also assume that AGI won’t operate under a game theoretic of winner takes all.

Monk.

Religion & Philosophy are going to be so important in the 21st century.

Joke written by Hank Hill.

The problem with large organisations is that they become slow. Generally startups are at the forefront of new innovation. It would take a shake up on the scale of Steve Jobs returning to Apple to get google to where it needs to be imo.

Yes, nothing seems more obvious to me. Google's whole value proposition was built on the PageRank algorithm (recommend reading the original paper), and 90% of it's revenue still comes from search. ChatGPT is like PageRank on steroids, it compresses information into knowledge, which is what people want when they google something.

It doesn’t need to be full AGI to be dangerous. As long as it is better than humans in some narrow setting it could be dangerous. Examples: Software companies like Palantir have shown that AI can determine who wins and loses a war, it has allowed Ukraine to outperform a larger country with more military might.

Then there are all the ways it can be used to sway public opinion, propaganda generation, and win in financial markets/financial warfare. And the one I’m particularly afraid of is when it learns to compromise computer systems in a cyber warfare scenario. Just like in a game of Go or chess, where it discovered moves that boggled the minds of experts at the game, I can easily see an AI suddenly gaining root access to any computer network it likes.

People have been downplaying AI for too long, every year it gets more powerful and people are still like “meh, still way off AGI super-intelligence!” and they probably won’t change their mind until an autonomous robot knocks on their door and drags them into the street.

We need to start thinking seriously about how this will play out and start preparing society and institutions for what’s to come. It’s time to sound the alarm.

It all ultimately works by sets of rules and laws which means that throwing more compute at it will eventually yield more secrets.

I wonder how much cool shit is going on behind the scenes that they don’t report to the media.

At some point in that gradual progression AI must reach a level that is equivalent to a human though right? Or do you think it just skips a few steps and goes straight to ASI?

So you’re inclined to think the hard takeoff scenario is more likely?

Finnegan’s wake. Might take a lifetime to decipher it.

There is some game theory at work here, the thinking is “If we don’t develop it our competitors will and outcompete us”.

I’m astonished how insecure, leaky and anarchic the internet is. I think a decade from now we will look back on the current internet as the Wild West: manipulation, hacks, spam, viruses, bots. Hopefully by then the internet will be a lot nicer place where people come to vote, work and socialise.