Viewing a single comment thread. View all comments

TemetN t1_j8gmwj1 wrote

Reply to comment by ThirdFloorNorth in Altman vs. Yudkowsky outlook by kdun19ham

To be fair, Yudhowsky's argument on Pascal's mugging was actually interesting (particularly vis a vis his own writings funnily enough), but yes I very much consider him someone you have to sort through the writings of due to his focus on foom and... well, pessimism is an understatement, but I hesitate to call him a doomer since most of them don't even have coherent arguments.

​

Altman is still something of a hypeman though, and it is worth noting that both of them have argued in favor of very agressive AI timelines, which has been generally more towards where things have actually occurred as compared to the preponderance of people expecting ridiculously slow progress.

10

gay_manta_ray t1_j8h0ys4 wrote

personally, i really dislike any serious risk consideration when it comes to thought experiments like pascal's mugging in regards to any superintelligent ai. it has always seemed to me like there is something very wrong with assuming both superintelligence, but also some kind of hyper-rationality that goes far outside of the bounds of pragmatism when it comes to maximizing utility. assuming they're also superintelligent, but also somehow naive enough to have no upper bounds on any sort of utility consideration, is just stupid. i don't know what yudhowsky's argument was though, if you could link it i'd like to give it a read.

8

TemetN t1_j8h21sz wrote

Reasonable. Honestly, I more found the premise interesting than the application, but it sounds like you've at least read one of the discussions about it. If not, here's the original (you can get to some of the others through the topic links up top).

Pascal's Mugging

4

bildramer t1_j8htdli wrote

It's not about naïvete. It's about the orthogonality thesis. You can combine any utility function with any level of intelligence. You can be really smart but care only about something humans would consider "dumb". There's no fundamental obstacle there.

1