Comments

You must log in or register to comment.

etesian_dusk t1_j9f86r8 wrote

The question is not related to /r/MachineLearning

15

ninjadude93 t1_j9f5024 wrote

Pretty sure you just got lucky examples cahtgpt isn't doing any logical reasoning. Everything it outputs is statistically generated not logically reasoned

8

vladosaurus OP t1_j9gncvj wrote

Yes, I see your point, thanks. Probably I got a lucky shot. In any case, ChatGPT was updates with math reasoning so I was just curious.

1

Delacroid t1_j9flphj wrote

Did you really need to numerically compute the gradient to check it was OK? Dude this is high-school math.

3

vladosaurus OP t1_j9gn4i0 wrote

Dude it's ok, I know it is high-school math you proved your point you are genius you know high-school math I don't.

That was not my aim. It was to treat the ChatGPT implementation as a black-box without touching it, and see whether is correct.

1

Delacroid t1_j9grwzu wrote

I'll admit that my comment may come off as elitist, but I think that you have to admit that this was a very low effort post. Maybe a more correct sub for this post would have been r/learnmachinelearning.

1

vladosaurus OP t1_j9gu3cr wrote

Ideally we have to generate many examples as such without seeing them and wrap them in some test suite using automatic differentiation to see how many will come out correct.

Something similar to what the authors did in the OpenAI Codex model. They provided the function signature and the docstrings and promted the model to generate the rest of it. Then they wrapped the generated function into test suites and calculated how many of them pass. It's the pass@K metric.

I am not aware if something similar is done for differentiation, maybe there is, I have to search for.

0

Delacroid t1_j9itr09 wrote

Well that good be an amazing post to read. How many times does it get math questions right but with an statistically significant number of samples. So that we can actually compare to the state of the art, such as galactica.

1

Kristitsope t1_j9fqywd wrote

These posts are getting out of hand

1

friend_of_kalman t1_j9g1k6t wrote

>ChatGPT seems to have good skills to calculate

No, it doesn't. It's a statistical language model, not a rule-based calculator.

1

vladosaurus OP t1_j9gnm8n wrote

In the recent update it was mentioned that the math skills were improved. So I was just curious to see. But thanks for your opinion.

1

blablanonymous t1_j9g840u wrote

“Can we use ChatGPT to use ChatGPT?” GTFO

0

vladosaurus OP t1_j9gnehk wrote

ChatGPT running on ChatGPT like a Turing Machine. GTFO!

0

vladosaurus OP t1_j9gnh7n wrote

Okay everyone, easy with the negative sentiment, I'm just experimenting with it, was curious to hear some opinion. But there it is I have it now .... nothing constructive in general except one comment.

0