Viewing a single comment thread. View all comments

RadioFreeAmerika OP t1_jdr3b6j wrote

Reply to comment by ArcticWinterZzZ in Why is maths so hard for LLMs? by RadioFreeAmerika

Thank you very much for your clarification! Do you know if it is possible to make a LLM with more space and greater complexity than O(1) or how it possibly could be added to GPT-4 with or without plug-ins?

1

ArcticWinterZzZ t1_jdt10ie wrote

Yes, it can probably be done. How? I don't know. Maybe some kind of neural loopback structure that runs layers until it's "done". No idea how this would really work.

3

liqui_date_me t1_jdr7pnr wrote

Tough to say, probably in 10-20 years at the very least. Modern LLMs are transformers which are architected to predict the next token in a sequence in O(1) time, regardless of the input. Unless we get a radically different neural network architecture it’s not possible we’ll ever get GPT to perform math calculations exactly

2

sdmat t1_jdut7jg wrote

Or just go with a workable hack for calculation like the Wolfram plugin.

Does it matter if the model isn't doing it natively if it understands how and when to use the tool? How often do we multiply large numbers unaided?

1