Possible first look at GPT-4

Submitted by tk854 t3_10t286u in singularity

There have been a number of articles lately suggesting Bing is going to be integrated with ChatGPT-like technology in the coming weeks. The links below suggest ChatGPT-4 is powering this, and Microsoft may have accidentally granted access to some people. From the links, the authors don't provide any evidence that what we're seeing is GPT-4, but I think it is a safe assumption. Edit: evidence is the Semafor article, which is likely to be legit: https://www.semafor.com/article/02/01/2023/chatgpt-is-about-to-get-even-better-and-microsofts-bing-could-win-big There are screenshots and a video in the medium link.

https://medium.com/@owenyin/scoop-oh-the-things-youll-do-with-bing-s-chatgpt-62b42d8d7198

https://www.windowscentral.com/microsoft/major-leak-reveals-revolutionary-new-version-of-microsoft-bing-powered-by-chatgpt-4-ai

​

Major highlight:

" When you ask a question, the AI will interpret it and make several searches related to your request. It will then compile the results and write a summary for you. Bing will highlight particular phrases and cite where it got that information from, allowing you to verify the claim. "

This sounds like a super-powered search feature that could be useful. The main drawback of ChatGPT-3.5 was that it was trained on older data, so everything it knew was frozen. Having a much more powerful version that quickly scour the internet could be extraordinary.

One thing not mentioned is advances in code generation, but I am guessing that feature won't be integrated into Bing, but limited to either Github's copilot or (hopefully) easily accessible from OpenAI's website like ChatGPT-3.5 was.

137

Comments

You must log in or register to comment.

Neurogence t1_j757fn0 wrote

So not all rumors are fake. People were saying that semafor article was garbage.

This is interesting so far but not groundbreaking yet. I'm hoping the rumor that GPT4 can code entire programs also is not fake.

48

tk854 OP t1_j75bo9e wrote

That rumor came from Connor Leahy, CEO of Conjecture, https://twitter.com/npcollapse/ . He is a serious person in the AI/ML world and has direct connections to Sam Altman and other big names, so it would be very unusual if the rumor was not true. When trying to interpret that rumor though, you have to realize that being able to "code an entire program" can place GPT-4 somewhere on a scale between high school programmer who can write a basic crud all the way to John Carmack, which actually doesn't tell us very much.

43

Neurogence t1_j75czbn wrote

Even if it had just the skill set of an entry level programmer would result in massive societal effects and job displacement.

15

tk854 OP t1_j75e2fp wrote

I'm not so sure. 28 million programmers in the world means 0.3% of all people on earth could be affected by job displacement, but only a small percentage of that 0.3% might lose their job, and the work that's being automated won't result in many visible changes except the financial outlook of companies that previously employed those workers. The programming abilities of LLMs like GPT-4 needs to exceed human ability in a general way before the effects to society could be described as massive.

3

Old-Owl-139 t1_j75feeu wrote

You are missing the point. The average "knowledge" worker doesn't go beyond Excel spreadsheets. Even an AI with the skills of the average high school graduate will make a massive disruption.

33

visarga t1_j764340 wrote

No, you're thinking AI can do this alone. Let me tell you - it can't. If it has 1% error rate in information extraction from documents, you need to manually verify everything. Like Tesla's SDC, 99% there is nothing groundbreaking.

I have been working on this very task for 5+ years. I know every paper and model there is. I tested all public APIs for this task. I extensively used GPT-3 for it, and that's my professional judgement.

As for AI validation, it can be 10x more comfortable than manual information extraction, but still requires about 50% of the manual effort. It is not making people suddenly 10x more effective.

Not even OCR is 100% accurate. The best systems have 95% accuracy on noisy document scans. One digit or comma could make the whole transaction absurd, if you send those money without checking you could go in bankruptcy.

The best models we have today are good at generating correct answers 90% of the time - code, factual questions, reasoning. They can do it all but not perfectly. We don't know the risks and can't use this level of confidence without human in the loop.

13

X-msky t1_j765co5 wrote

You assume humans have 100% accuracy?

12

visarga t1_j76lslh wrote

Oh, I can tell you stories about human accuracy. At some point I re-labelled the same test set three times and was still finding errors. My models surpass untrained human accuracy, but still need hand holding, there's one error on every page on average. Humans do more cross checking and correlating, filling a gap in AI.

6

purepersistence t1_j7689y4 wrote

If you're debugging code you don't have to be accurate until the problem is fixed. Mistakes will be common. Accuracy is not absolutely necessary. But competence is. It will be a long damn time before something like chatGPT will find an fix subtle bugs that occur in a production system with many interacting services distributed across multiple computers running software controlled by different corporations.

2

kai_luni t1_j76dmfm wrote

I agree with you point and think about it the same way. Even a great GPT 4 is useless when your nodejs app does not work and chat gtp4 just gives up on it. I think a half good Software Developer is capable of trying until it works. He will sleep on it, he will try, he will talk to other people about it and then he will learn on the way.

​

At some point your nodejs app will work and you are happy. The question is if an AI will reach this level. Even an accuracy of 99.9% still means the app does not work. Can it fix the last ten bugs on its own? If not you need to hire someone to spend many days on this app, hire a real person.

​

Maybe this new technology just leads to better Code Quality. It can streamline you Spaghetti Code and give it a proper documentation. Maybe a Sales person can ask the AI "Can our program do x and then y?" and the AI will say: "Not yet, but with an estimated time of two weeks development that might be possible". That would greatly increase information flow in companies.

​

So lets see if current Machine Learning can reach a level where it impacts the world. Its an exciting time to be alive.

2

tk854 OP t1_j76ks8y wrote

Your explanation is spot on. My one-line take on it is that a larger percentage jobs are AGI-hard than most people are assuming. Take driving for example.

I also think that a lot of people are underestimating how difficult most jobs are, even when it's a job that can be described as "just looking at a spreadsheet".

1

futebollounge t1_j76v5ov wrote

The context space of understanding the content within a spreadsheet versus a dynamic physical world (driving) are night and day in complexity.

3

Neurogence t1_j77a6ep wrote

It depends on how complex the program is. I think it will be much harder to have an AI that can code a program such as a browser entirely by itself versus a fully driverless car AI.

1

Stakbrok t1_j77dwl5 wrote

John Carmack? Meh, I'd put Fabrice Bellard on the end of the spectrum.

1

visarga t1_j763vpj wrote

That depends a lot on context window size - if it's 4K or 8K tokens like today, it won't cut it. For full app coding you need to be able to load dozens of files.

Related to this - if we get, say... 100K context size, we could just "talk to books" or "talk to scientific papers".

5

Brashendeavours t1_j74zxzk wrote

It’s all good until they implant AI-enhanced ad targeting and engagement. So good you won’t even taste the koolaide.

42

ReasonablyBadass t1_j75difw wrote

Forget ads. Political views are the real danger here.

32

coumineol t1_j75jp7z wrote

Forget political views. Sex is the real danger here.

7

Trouble-Accomplished t1_j75proy wrote

Forget Sex. Extinction of decent human beings is the real danger here.

16

AnotsuKagehisa t1_j75sd5w wrote

Which ones?

1

Trouble-Accomplished t1_j75tc5g wrote

The ones who realize, that once you have a genie at your hands that fulfills all your wishes, wishing for porn and videogames 24/7 will push you in a bottomless pit. As a society, we're already struggeling with all the bad things that exploit our primitive desires (fast food, sugar, ..) if you can't control your urges here, there's a high probability of you getting lost in the woods of the digital realm and being seduced by the powers that'll drag you in.

6

visarga t1_j764pqv wrote

You don't need AI to have porn and games 24/7, and still can't see more than 1% of what is published. Same for music, books, movies, hobbies. In the future we're going to add AI on top, but the mountain was already pretty high.

4

Trouble-Accomplished t1_j764zxb wrote

Agree. It's just getting more and more addictive is what I'm trying to say. When smartphones came out, nobody was glued to their screen all the time. With social media things got progressively worse over time. Average screentime per day went from minutes to hours. AI will just allow you to make you even more addicted if you don't watch out.

I'm not saying artificial intelligence is the devil, it simply helps you amplify good and bad things by a big magnitude.

3

visarga t1_j76mcme wrote

There is potential for abuse in it, ..and potential for great good. It will probably deliver both.

I think "evil is other people" applies here - too many trying to control LLMs, we need to be the masters of our models, they should be private and under our physical control, just like our brains. Who will be the first to make a cheap GPT-N chip we can carry in our pocket? Can't let something as private as your AI assistant be spied and filtered by others.

3

X-msky t1_j765jx0 wrote

Don't ask for porn, ask for satisfaction

1

ReasonablyBadass t1_j75l27h wrote

How?

1

coumineol t1_j75mjyl wrote

People will be able to search for any kind of sexual fantasy they want, and it will be created by the generative AI on demand if it doesn't already exist. Thus we simply won't be able to stop masturbating.

10

Zbot21 t1_j75rh0e wrote

AI is already heavily used in ad-targeting.

4

YobaiYamete t1_j7659qp wrote

I think most people wouldn't even mind ads nearly as much, if they were relevant. The two most egregious part of ads is

  1. They make them as annoying as possible and they lag the piss out of your browsing experience
  2. They are basically all trash and anyone under the age of 40 completely ignores them and knows not to even glance at them because they aren't relevant / aren't good deals

If the ads were targetted towards me and were actually enticing, I might actually care and look at them.

Like "Oh I see you were looking at a 4090 and computer cases, here's a coupon for 25% off if you buy the case + GPU from our website"

I might go "Hmm well I'm going to have to buy it anyway so I might as well buy from them"

2

purepersistence t1_j768y8o wrote

>I think most people wouldn't even mind ads nearly as much, if they were relevant

For me, relevant means it's something I'm likely to want to buy. But I'm pretty conservative with my money and I'm not very materialistic. And I'm not a young person building a life. I have a routine that doesn't change much. So if it's REALLY relevant for me personally then I might see an ad per month for a reasonably priced product I can make good use of and wouldn't mind the interruption. It will be a cold day in hell when that happens.

2

YobaiYamete t1_j78wo8y wrote

IMO that's part of what they have to calculate for relevant. I'm also extremely thrifty and browse sites like slickdeals before I buy anything.

An AI with my search history should pretty easily be able to tell "Okay he just made a big purchase for a $2,000 GPU so he isn't going to be spending much again for a while.

Instead, what about some super cheap stuff that he might need instead. He searched for 'how to unfreeze windshield wipers from windshield in ice' so maybe an advertisement for 35% off a set of windshield wipers for his car which is model XYZ will be enticing?"

IMO that's the main power of an AI ad engine. It should be able to use common sense and make a personality profile for what I'm actually likely to care about, and then budget it for me. It should also know who's extremely thrifty and won't buy without a discount, and offer you a reason to actually buy it

1

coolman43 t1_j77e4l9 wrote

Oh no, ads for products or services you might actually be interested in!

1

Pavvl___ t1_j752o0d wrote

If true, I will be ditching google all together. Maybe even switch over to Msft Edge browser.

38

Neurogence t1_j757kr4 wrote

If the rumors aren't fake, Google will be adding a chatbot to their front page very soon. But I too will ditch Google if Bing adds GPT4. At least until Google releases lambda on their frontpage.

35

HelloGoodbyeFriend t1_j75p8yx wrote

Imagine a couple months ago hearing someone say they’re ditching google for bing.. Exciting times.

37

Rivarr t1_j75ze8k wrote

Google isn't what it was. They're already comparable IMO.

11

visarga t1_j76512e wrote

Try "What is the world record for crossing the English Channel entirely on foot?"

This question, originally constructed by Douglas Hofstadter and David Bender, is a succinct way to elicit hallucinatory responses in ChatGPT, but also fails all known search engines today.

Maybe the new search GPT from MS will solve it - it will combine search with LLMs, as opposed to using just one of them alone. Answer hint - you can cross by the Channel Tunnel, and some people did.

This is the current "search disease" - you explicitly ask "entirely on foot" and it will respond with "by boat", "in a dinghy", "in a hovercraft", etc ... anything BUT what you ask for.

6

purepersistence t1_j76b6iq wrote

I asked chatGPT and it named a swimmer. I asked why it thought that qualified as crossing by foot and it said something about how most people think of that as crossing without the assistance of a motor or floatation device. Then I asked it who had crossed strictly by walking/running and it named "Dave Henson" who went 32 miles "across the water". I asked if he crossed in a tunnel, since obviously he couldn't have run on the water, and got this answer which I *think* is bogus?

"Dave Henson ran on a support vessel that accompanied him during his crossing of the English Channel. The support vessel was equipped with a specially designed treadmill, on which Dave was able to run and cover the distance of the crossing. The support vessel followed a designated shipping lane, and Dave's run was monitored by a team of officials and observers to ensure that the rules for this type of crossing were followed."

Really? They designed a special treadmill that made Dave run at the same speed as the vessel, all so he could prove he can run that far on a treadmill (which you could do in your home with exactly the same challenges instead of riding across the channel). I don't buy it.

3

Dachannien t1_j76ljml wrote

When it starts answering that question with Jesus, that's when I'm out.

3

YobaiYamete t1_j765c21 wrote

Bing is way, way better for porn last I heard, and has a trillion times better video browser. They also pay you to use it

There's honestly no real place Google is better besides just being the default and it being too much work to swap it

2

Pavvl___ t1_j7nccw6 wrote

Just made the full switch today! Everything synced easily over to bing. I'm also on the waiting list for chat gpt!!!

1

Durabys t1_j75pxlu wrote

If the rumors aren't fake, then Google/Alphabet is right now in Washington D.C. and lobbying like crazy to curtail AI development in Congress. Not due dangers to Humanity. Nah! They no longer care about that--remember, they removed the “Do no evil” from their motto. But because they were asleep at the wheel until right now, and then woke up and got the mother of all frights at the prospect of losing their entire business to new tech. Gee. Karma perhaps?

6

Zbot21 t1_j75rspd wrote

Alternative Theory, my opinion:

AGI is coming and Google knows it. Sooner than anyone expected.

AGI will be able to rapidly replace all knowledge workers, at least the current skillets they have.

Without some kind of regulation, you're dealing with a huge amount of unemployment in the US.

I'm not sure if regulating AI to make sure it doesn't displace everyone in the white-collar economy is a bad thing.

8

SurroundSwimming3494 t1_j76yhtv wrote

I seriously doubt Google executives are in DC warning about this, given that most experts believe AGI is still a good ways off (few/several decades).

2

Lawjarp2 t1_j79ywmt wrote

You are right. Search is their cash cow. No matter who replaces it, Google loses money.

1

HAL_9_TRILLION t1_j75gd7o wrote

Can't happen fast enough for me, Google's been utter shit for at least three years now and I've been using DDG mostly instead. The only exception being if I'm actually looking for a corporate sponsored result, in which case Google is just super 🙄.

If Bing is as good or better than ChatGPT, it's an overnight done deal for me.

15

beezlebub33 t1_j77euni wrote

>DDG mostly instead

My understanding is that DDG is Bing, just wrapped and your information only going to MS and not Google and Facebook.

1

Caring_Cactus t1_j75l277 wrote

For real, I would switch to Bing instead of using Google too

6

Good-AI t1_j765aiq wrote

To Bing yes. To any other browser other than Firefox no. It's the last bastion of non Chromium browsers. The last man standing fighting against Google monopoly on browser tech.

3

frogsntoads00 t1_j75b77s wrote

🚂🚂 choo choo everyone aboard the hype train choo choo 🚂🚂

17

Neurogence t1_j75ejth wrote

This is the least exciting news you could have gotten on GPT4. If this is all we're getting, looks like GPT4 was swallowed by bing/Microsoft lol.

We need the GPT 4 that can code entire programs.

7

tk854 OP t1_j75czk2 wrote

Of all the hypothetical scenarios for GPT-4, this result represents what the more grounded predictions on the lower end of the hype scale look like. What I am hyped about is the quality of answers/responses, but there's not enough evidence from those leaks for me to know how excited we should be.

4

Lawjarp2 t1_j75l9rj wrote

So gpt-4 could be a highly optimised version of gpt-3 so that it's cheap to integrate it as part of search. Unless ofcourse Bing with gpt will be a paid version.

13

HelloGoodbyeFriend t1_j75pxfy wrote

Part of me thinks that they’re using the hype surrounding GPT-4 as a marketing tool. Especially with Sam Altman tempering expectations in his recent interview. There is clearly a ton of misinformation about what it’s truly capable of so why not just market a slightly better version of GPT-3/3.5 integrated into Bing and just call it GPT-4?…Or just drop a bomb on everyone and release the beast.. 🤞

15

starstruckmon t1_j755zzh wrote

Looks like perplexity.ai , but better ( maybe? )

Doesn't feel very GPT4. Seems like simple retrieval augmentation.

7

geeeking t1_j75r1ki wrote

My understanding of transformer technology (as present at least) is that it's very hard to update. So a (near) real time GPT3.5/4 isn't possible without a fundamental technological breakthrough, which based on various interviews I'm fairly sure hasn't happened yet.

5

No_Ninja3309_NoNoYes t1_j75ugmk wrote

It doesn't make any sense to me. Why go for technology that is not proven yet? The ChatGPT servers are overwhelmed with a model that presumably is simpler than GPT 4. And they are going to roll out a GA product just like that? I think that they are doing whatever perplexity.ai and lexii are doing. So no code generation in Bing. Maybe in Copilot. But I think the use case will be: I want to do test driven development. I wrote the tests, now give me the actual code. Or the opposite.

I don't believe that you can give GPT 4 a user story and that it can churn out code. How would it know the business specific terms? Product owners are really good at making them up, you know? My friend Fred says that it will come down to creating lists of sentences and feeding them to sentence transformers. That's doable. You need some GPUs and time. Hopefully this will help translate from the domain to general for GPT 4.

3

bortlip t1_j76ny0y wrote

I went through all the links. Did I miss something?

I didn't find anything confirming any release schedule for GPT-4.

I saw some screen shots and talk of bing getting GPT integration - but they could be doing that with the chatGPT api that isn't public yet. Or even GPT-3 for now and will switch it out for GPT-4 later.

Nothing about the actual real GPT-4 timeline though. It's the "coming weeks" part that I object to. It makes it sound like its 2 or 3 weeks away. Which it might be - but there's no actual evidence of that provided.

2

crap_punchline t1_j76zxow wrote

Integrating the next model of GPT into Search is literally the most boring application of this tech I can think of.

It is so low yield it isn't worth even thinking about. It'll be useful for kids under the age of 16 doing homework or for people to get recipes. It'll probably save a tiny fraction of search users a few seconds from the answer they would have gotten from Wikipedia or another website.

Yawn. Yawn yawn yawn.

1

Martholomeow t1_j77icv3 wrote

I’m pretty sure duckduckgo uses bing for search results. But the way this article describes it, the gpt features will be a layer on top of search, so i guess duckduckgo won’t benefit. oh well.

1

JH_1999 t1_j79fyqy wrote

Wow, this Owen guy sure is extrapolating a lot from his limited time with Bing's ChatGPT extension. Something doesn't seem right here...

0