huberloss t1_iz8kqp3 wrote
I asked ChatGPT this question. This is what it came up with (I am very impressed at what it output):
There have been many significant papers on artificial intelligence (AI) published over the past 100 years. It would be difficult to narrow it down to just 10-20 papers, but some notable papers that have helped shape the field of AI include:
"A Logical Calculus of the Ideas Immanent in Nervous Activity" by Warren McCulloch and Walter Pitts (1943) - This paper introduced the concept of a "neural network" as a computational model for the workings of the brain.
"Computing Machinery and Intelligence" by Alan Turing (1950) - This paper introduced the Turing test, a benchmark for determining whether a machine can exhibit intelligent behavior.
"Perceptrons" by Frank Rosenblatt (1958) - This paper introduced the concept of the perceptron, a simple mathematical model of a neural network.
"Artificial Intelligence: A New Synthesis" by Nils Nilsson (1998) - This book provided a comprehensive overview of the state of the art in AI research at the time.
"The Logic Theorist" by Allen Newell, J. C. Shaw, and Herbert Simon (1956) - This paper described a program called the Logic Theorist, which was able to prove mathematical theorems using artificial intelligence techniques.
"Grammar Induction and Parsing with a Recursive Neural Network" by Stephen Clark and James R. Curran (2007) - This paper introduced the use of recursive neural networks for natural language processing tasks.
"A Survey of the Monte Carlo Method" by Alan Gelfand and Adrian Smith (1990) - This paper provided an overview of the Monte Carlo method, a computational technique that has been widely used in AI.
"The Elements of a Scientific Theory of Intelligence" by Judea Pearl (2000) - This paper introduced the concept of causality, which has become a key focus of AI research.
"Hierarchical Temporal Memory" by Jeff Hawkins, Dileep George, and D. S. Modha (2004) - This paper introduced the concept of hierarchical temporal memory, a computational model for the workings of the brain.
"Human-Level Control through Deep Reinforcement Learning" by Volodymyr Mnih, et al. (2015) - This paper introduced the use of deep reinforcement learning for achieving human-level performance in a range of challenging tasks.
These are just a few examples of some of the significant papers in the field of AI. There are many others that have contributed to the development of AI over the past 100 years.
anonymousTestPoster t1_iz8pwzx wrote
Schmidhuber would like a word with that ChatGPT bot
Tejas_Garhewal t1_iz9ormz wrote
Who is this Schmidhuber guy, seeing him getting mentioned quite a bit
pacific_plywood t1_iz9pesa wrote
He is an extremely prolific researcher who believes that his lab was the first to publish on a number of significant topics
Screye t1_izauw5w wrote
He is the UIUC of Deep learning's mount rushmore.
Just as people think of Stanford, MIT, CMU, Berkley as the big CS universities and forget that UIUC is almost just as good.....people take the names of Hinton, LeCun, Bengio and forget that Schmidhuber(' lab) did a lot of important foundational work in deep learning.
Sadly, he is a curmudgeon who complains a lot and claims even more than he has actually achieved.....so people have kind of soured on him lately.
undefdev t1_izbjvcl wrote
> Sadly, he is a curmudgeon who complains a lot and claims even more than he has actually achieved.....so people have kind of soured on him lately.
What did he claim that he didn't achieve? I didn't dig too deeply into it, but it always seemed to me that his complaints haven't been addressed, but nobody has an incentive to support him.
JustOneAvailableName t1_izbnfki wrote
> What did he claim that he didn't achieve?
Connections to his work are often vague. Yes, his lab tried something in the same extremely general direction. No, his lab did not show it actually worked or what part of the broad direction they went in actually worked. So I am not gonna cite Fast Weight Programmers when I want to write about transformers. Yes, Fast Weight Programmers also argued there are more ways to handle variable sized input than using RNNs. No, I don't think the idea is special at all. The main point of Attention is all you need was that removing something of the then mainstream architecture made it faster (or larger) to train while keeping the quality. It was the timing that made it special, because it successfully went against mainstream and they made it work, not the idea itself.
undefdev t1_izbui6y wrote
> So I am not gonna cite Fast Weight Programmers when I want to write about transformers.
I think you are probably refering to this paper: Linear Transformers Are Secretly Fast Weight Programmers
It seems like they showed that linear transformers are equivalent to fast weight programmers. If linear transformers are relevant to your research, why not cite fast weight programmers? Credit is cheap, right? We can still call them linear transformers.
JustOneAvailableName t1_izbzbaq wrote
Because Schmidhuber claiming that transformers are based on his work was a meme for 3-4 years before he actually did that. Like here.
There are hundreds more relevant papers to cite and read about (linear scaling) transformers
undefdev t1_izc3tr1 wrote
> Because Schmidhuber claiming that transformers are based on his work was a meme for 3-4 years before he actually did that. Like here.
But why should memes be relevant in science? Not citing someone because there are memes around their person seems kind of arbitrary. If it's just memes, maybe we shouldn't take them too seriously.
vwings t1_iz8n5m2 wrote
i would add:
LSTMs (1997), Hochreiter & Schmidhuber
ImageNet (2012), Krishevsky et al
Deep Learning (2015), LeCun, Hinton & Bengio
Attention is all you need (2017).
FutureIsMine t1_iz8p67d wrote
Schmidhuber has entered the room and demands he be acknowledged for chatGPT
RobbinDeBank t1_iz9mrjg wrote
chatGPT is actually a specific case of the general learning algorithms introduced in Schmidhuber et al. (1990)
Garbage-Shoddy t1_iz9bq1v wrote
Typical Schmidhuber
Noddybear t1_iz8vddc wrote
Don’t forget Cybenko’s 1989 Universal Approximation Theorem paper.
robbsc t1_iza17he wrote
I don't think the deep learning paper is really significant. It just brought attention to recent advances.
vwings t1_iza8lwm wrote
Completely agree.
SleekEagle t1_iza258i wrote
Agreed - the last three you listed were the first ones that came to mind for me
Nameless1995 t1_iz92pe9 wrote
> "Grammar Induction and Parsing with a Recursive Neural Network" by Stephen Clark and James R. Curran (2007) - This paper introduced the use of recursive neural networks for natural language processing tasks.
Is this one hallucinated? Couldn't find it.
Some other seems hallucinated too, although semantically related to kind of things the authors do.
MrAcurite t1_izagr44 wrote
The 'C' in 'ChatGPT' stands for "Confident Bullshitting."
The 'hat' identifies this as merely an approximation of confident bullshitting.
huberloss t1_izadd9p wrote
>Grammar Induction and Parsing with a Recursive Neural Network
Pretty sure it's hallucinated. It's kind of funny how plausible it sounded, though :D
versaceblues OP t1_iz8mep7 wrote
interesting I asked gpt the questions as well... it gave me a slightly different set
rsandler t1_izghl8o wrote
Alot of these paper titles are hallucinated.
For example, I couldnt find:
"Grammar Induction and Parsing with a Recursive Neural Network"
"A Survey of the Monte Carlo Method"
Also, interestingly, Pearl never wrote a book called "The Elements of a Scientific Theory of Intelligence", but in 2000 he did write his seminal "Causality: Models, Reasoning, and Inference" for which the description would apply very well to...
Viewing a single comment thread. View all comments