Viewing a single comment thread. View all comments

semitope t1_j8idqk3 wrote

Reply to comment by LordKeeper in ChatGPT Passed a Major Medical Exam, but Just Barely | Researchers say ChatGPT is the first AI to receive a passing score for the U.S. Medical Licensing Exam, but it's still bad at math. by chrisdh79

>without needing to acquire additional material from the internet or elsewhere

It doesn't constantly search the internet to come up with it's answers? It needs data. All software needs data. Not sure how it works but its either it has access to the internet to look through it and uses indexing like google, or their servers have stored massive amounts of data for it to be relevant in different areas.

I doubt AI can do well in fact heavy fields like law and medicine with no way of knowing the facts.

−4

GondolaSnaps t1_j8in58c wrote

It was trained on massive amounts of internet data, but it isn’t online.

If you ask it, it’ll even tell you that all of it’s information is from 2021 and that it has no knowledge of anything after that.

For example, if you ask it about Queen Elizabeth it’ll describe her as the current monarch as it has no idea she’s already dead.

9

MilesGates t1_j8jehab wrote

>It was trained on massive amounts of internet data, but it isn’t online.

Sounds kind of like doing an open book test where you can read the textbook to find the answers but you can't google for the answers.

1

jagedlion t1_j8jxe3s wrote

Common misconception. It memorizes the data and forms connections in its model. It's sort of like memorization in that way, as it doesn't even store any of the raw information it was trained on. It only stores the predictive model.

This is also why you can implement AI vision algorithms on primitive microcontrollers. They don't have the computational power to solve for the AI model, but once the powerful computer calculates the model, a much simpler one can use it.

2

semitope t1_j8k09qi wrote

sounds about the same thing. given the data before vs looking for it now. Fact is it cannot produce useful responses when it comes to facts without exposure to the data. Would be like someone talking about something they know absolutely nothing about. Which might be why sometimes it's accused of making things up confidently.

0

jagedlion t1_j8k0uqa wrote

I mean, humans can't either give you information that they don't have exposure to. We just acquire more data during our normal day to day lives. People also do their best to infer from what they know. They are more willing to code their certainty in their language, sure, but humans also can only work off of the knowledge they have and the connections they can find within.

4

semitope t1_j8k5n5n wrote

humans aside, saying it doesn't need to acquire additional information from the internet or elsewhere isn't saying much if it already acquired the information from the internet and elsewhere. It already studied for the exam

0

jagedlion t1_j8kbruo wrote

Part of model building is that it compresses well and doesn't need to store the original data. It consumed 45TB of internet, and stores it in its 700GB working memory (the inference engine can be stored in less space, but I cant pin down a specific minimal number).

It has to figure out what's worth remembering (and how to remember it) without access to the test. It studied the general knowledge, but it didn't study for this particular exam.

2