Viewing a single comment thread. View all comments

adventuringraw t1_jdrnbmp wrote

Um no (we have to keep the comment chain going).

You're actually being overly dismissive of what they're saying I think. The key word they used was 'inspired'. I tried to dig up the origin of convolutional image kernels, and while I couldn't find much in five minutes of digging, I'm sure you're right, that they predate deep learning certainly, and possibly even digital computing entirely given that their historical origin was probably in signal processing.

Their comment though wasn't whether or not CNNs directly imitate biology, or that the way they did it was entirely novel... They were just pointing out that biology was an inspiration for trying it this way, and that part's unambiguously true. To my knowledge, the first paper introducing the phrase 'convolutional neural network' was from Yann LeCun. This one I believe, from 1989. If you look at the references, you'll note Hubel and Wiesel's 1962 paper introducing a crude model of biological vision processing is in the references. More importantly, Fukushima, 1980 is referenced (and mentioned in the text as a direct inspiration). This 'Neocognitron' is generally accepted to be the first proto-CNN. The architecture is a bit different than we're used to, but it's where things started... And as the author puts it in the abstract:

> A neural network model for a mechanism of visual pattern recognition is proposed in this paper. The network is self-organized by "learning without a teacher", and acquires an ability to recognize stimulus patterns based on the geometrical similarity (Gestalt) of their shapes without affected by their positions. This network is given a nickname "neocognitron". After completion of self-organization, the network has a structure similar to the hierarchy model of the visual nervous system proposed by Hubel and Wiesel.

So... Yes. CNNs weren't inspired by cow vision or something... Hubel and Wiesel's most famous work involved experiments on kittens. but CNN origins are unambiguously tied into Hubel and Wiesel's work in biological visual processing, so the person you're responding to is actually the one that was right. I just noticed even, some of the papers referenced from Wikipedia that you said didn't show biological inspiration are the same ones I mentioned even, so they were the correct papers to cite.

If I may be a bit rude for my own Sunday morning amusement: 'Thanks for being interested, but there is a lot of fluffery in ML discussions.'

Seriously though, it's an interesting topic for sure, and historical image processing techniques are certainly equally important to the history of CNNs... They were the tool reached for given the biological inspiration, so in all seriousness you're not entirely wrong from another perspective, even if you're not justified in shooting down a biological inspiration.

7