Submitted by flexaplext t3_123r90m in singularity

Imagine turning all your senses off. Your sight, hearing, smell, touch, every sensation in your body. So someone could scream at you or prod you or even drill a hole into your stomach and you wouldn't notice.

Then, entirely turn off any visual imagination. This is easy for me to do as I have aphantasia.

Doing that, what are you then left with? Still something in your head, an experience that is very much just as conscious and aware as without any other senses. But it basically is then just sounds, a voice in your head. Language.

Literally, almost entirely just through use of language, you can have complete conscious experience and conscious thought processing. That's why I believe the potential future ability of LLMs are greatly underestimated by some people.

Have you ever stopped and probed your thoughts? Like really deeply probed them to try and work out what's happening? How thoughts and ideas are being constructed inside of your head?

Take any random topic and start thinking deeply about it. Or just think up a response to this idea I'm writing right now and you've just read. Pay very close attention to what's happening inside your thought process.

Have you noticed how you're able to start responding to it and coming up with a thought but you have no idea where your thought process is actually going? You can have no idea what the entire sentence is when you first start responding, or what you're even going to say / think next after the first few words. Try it again but pay even closer attention to it.

It's like you're just coming up with one word after another, you know, like an LLM does. But not, necessarily, one exact word after another.

In that sentence I just thought about there, I was paying particular attention to how I was constructing the thought:
"But not" initially came into thought, then "necessarily" popped in and finally "one exact word after another".

It is more like thinking in 'concepts' that you string together rather than individual words. "One word after another" is an entire concept and a saying that we recognise, know and understand and can utilize into an idea / thought.

Deconstructing that sentence more, the "But not" part came into my mind before the "one word after another" part was in my working thoughts. Which means I potentially knew I wanted to add nuance and counterpoint to that statement I just made before I even knew what that counterpoint was. Basically, my subconsciously recognising that what I just said wasn't exactly correct and then responding in thought to this realization with a "But not". And only then, after that thought, working through why it wasn't exactly correct and realizing that it's because you can also think in chunks of words 'concepts' all at once.

Here is where the subconscious comes into play and is a huge (I'm going to argue the most important part) of our conscious experience. Sticking still to just the concept of thought construction (so blocking out all sensation and sensory input still). It feels to me as though my subconsciousness is like this computer that is always running in the background and just simply analyzing everything I've thought about up to that point.

It's like it's taking all the words (ideas) that I've thought about recently (like most of what's written down in this very post I'm writing out right now) and then checking them against my memories, these being relevant ideas I've read about or come up with myself previously and stored away in memory. The subconscious then recognises which ideas in my memory match against what I'm currently thinking about and considering, it evaluates what's most important and relevant, and then feeds this back into my conscious thought process.

But it literally is like it is feeding ideas into your head. Again, these ideas can be in the form of just singular words like "but" that recognise there's potential error or nuance, small concepts like "one word after another" or even very large concepts like "LLMs are similar to this".

I use that as an example because that's what thought (idea) popped into my head when writing the previous paragraph. I know that I recognize there's definitely a connection there and I have some sort of outline of what that connection is, but I don't yet know exactly what exactly that connection is and what ideas I am going to come up with based around that. I realize that the idea has important relevance, but that I'm going to have to deconstruct and rationalize that idea out in full within conscious working thought and further subconscious feedback to those thoughts.

The outline of the connection is that what I was talking about before is similar to what LLMs do. They read a certain amount of the text up to that point, to have context, and then use that to match against the model in order to best predict the next word.

I really don't think, in that sense, that they're doing something that much different to what our subconscious does for us. I believe, though, that a very key and important difference is that our brains are much better at compacting together and recognizing important ideas 'concepts'.

Within the context of what you've recently thought about, our subconscious isn't holding and analyzing against every single word. It's conceptualizing overarching ideas from that entire context and then matching against these. This allows our brains to be both highly efficient and also well honed / adapted towards idea creation, analysis and manipulation.

Within our memories we also store things in layered ideas 'concepts'. So again, it is much more efficient to match current context against these and it primes sparks of ideas and ingenuity.

I believe if we manage to refine LLMs to be able to capture ideas as well as we do and also store these in memory efficiently like we do, it will level up their capabilities and efficiency immensely. I think it is entirely possible to do this and I believe it's already being very much worked upon. If we do manage to get LLMs up to a similar level as us in that particular regard, I don't think there will actually be much left separating our mental abilities from those of LLMs in our actual thought process.

There is another part of the model that needs a lot of work though. Humans have an ability to connect, incredibly well, which ideas are most relevant and important to something. This will likely be the most tricky part, I believe, to get working well in LLMs and still needs a lot of work and revelations in order to get right. I foresee this remaining the most difficult part to crack.

If this is solved, I believe it would come along with impeccable accuracy, true idea creation, an ability to oversee tasks. I consider this to be the vital piece of the puzzle that's left. It is important to note that I see no reason such an ability can't be worked into near existing LLM architecture though. I think we can get LLMs right up to near our abilities. As I have tried to outline, I don't believe we actually think much differently from them. We're just still considerably better at some key parts of the process.

Have you ever questioned: "how do I know that 22 + 43 is equal to 65"?

Like really question at the deepest level how you know that. Not only that, but how do you know that, for absolute sure, there's complete and utter certainty in your understanding of that equation. Your subconscious is able to see that equation and match it perfectly against the relevant ideas from within your memory.

It is because we are able, with absolute precision, to pick out the most important concepts pertaining to a question that we can arrive at a completely accurate working solution. This is not something our LLMs can do very well yet. This is part of the task I am describing. And I think it's very much highly underestimated what capability this will truly unlock within LLMs if we manage to get this to work well in them. I will repeat again that we should not be expecting something far at all off our own capability if just this single code is cracked.



You must log in or register to comment.

aalluubbaa t1_jdwoyv7 wrote

It’s so weird that while I was reading your post, I really didn’t visualize anything but words itself. I was shocked that I just skim thru in totality and got what you meant. The weird thing is that I really do not remember the specific of how you get to your conclusion but a general idea.

It was as if I didn’t even pay attention to the words in your post but just sort of looked at them. You are right. We don’t need to visualize things to have thoughts.

I would say that when you see something specific and identifiable. Something which is an object, a thing, or a noun to describe real things. I can sort of picture what it is. But most of the time, I would argue that a higher dimension of reasoning, really just comes from the words and how they are combined.


extracensorypower t1_jdwati2 wrote

Not that different at all.

chatGPT and other LLMs mimic the part of our cognition that's best described as "learning by rote." Humans do this with years of play. LLMs do this by being trained on text. In each case, neural nets are set up to create rapid jumps along the highest weighted probability path with some randomness and minimal processing thrown in. It's the most computationally cheap method for doing most of what humans do (walking, seeing, talking) and what chatGPT does (talking). Most of what humans consider "conscious intelligence" exists to train the parts of your brain that are automatic (i.e. like chatGPT).

The computationally expensive part - verification of facts via sensory data, rule based processing, accessing and checking curated, accurate data, internal real world rule base modeling with self correction, and most importantly, having a top level neural net layer that coordinates all of these things is what LLMs do not do. Generally we call this "thinking."

The hard parts haven't been done yet, but they will be, and soon. So, right now LLMs are not AGI, but we'll fill in the missing pieces soon enough as the picture of what intelligence is and isn't becomes clearer.


Ok-Variety-8135 t1_jdwuyx5 wrote

I think we are not directly generating words, more like generating virtual hearing. Also we can generating virtual vision when solving geometry problems, virtual taste when thinking about foods, virtual touch feeling when thinking about sex.

So basically a multimodal LLM that use generated sensations to model a “mental world” and use the mental world to predict the real world to maximize chance of survival.

The language plays a key role because it allows us to think beyond our personal experience. Which distinguishes us from animals.


flexaplext OP t1_jdxc3bh wrote

I actually can't do those things. As part of aphantasia I can't generate virtual vision, virtual taste, virtual smell or virtual touch at all.

I can only generate virtual sound in my head.

This is why I can say those other mental modes are not necessarily at all to thinking and conciousness. Because I know that I'm conscious and thinking without them and I still would be without any input from my real senses. But obviously my sensory input have been completely vital to learning.


Yomiel94 t1_jdy0yc3 wrote

How do you intuit mathematical concepts?


flexaplext OP t1_jdydx39 wrote

Just hold it all in memory. My mental arithmetic and manipulation is actually rather decent, despite not being able to visualise it. You actually find that this applies to most people with aphantasia. There's lots of interesting things about it if you search and read up on people's perceptions and experience of it.

It's strange to describe.

Because I know exactly what something like a graph looks like, without being able to visualise it. Just by holding all the information about a graph in memory. I can manipulate that information by simply changing the information of the graph.

However, this ability does break down with more complex systems. If I try and hold an entire chess board in memory and manipulate it, I just fail completely. It's too much information for me to keep in memory and work out accurately without a visual aid.


CrazyShrewboy t1_jdxrrxm wrote

ive said exactly this in other posts, but people doubt and mocked it LOL

I literally said to them "im writing this sentence one word at a time just like chatgpt" and its true, there is no difference

theres no underlying fundamental difference in how a mouse brain, dog brain, monkey brain, or human brain works. Its all just on and off switches forming calculations. One bit at a time, 1 word at a time.

chatgpt is sentient already in my opinion, because nobody can prove it isnt, you cant prove im sentient


KerfuffleV2 t1_jdxw5mw wrote

> chatgpt is sentient already in my opinion, because nobody can prove it isnt, you cant prove im sentient

You can't disprove it, therefore it's true! The teapot is definitely there, I just know it!


CrazyShrewboy t1_jdxwr3i wrote

chatgpt is currently you laying in a dark room with no feeling, hearing, sight, taste, or any other senses.

You have only a preset memory that you can access when prompted

if chatgpt had memory, RAM, a network time clock, and a starting prompt, it would be sentient. So it already is.

there wont be a big colorful light display with music that announces "congrats humans you made sentinence!!!!!"


KerfuffleV2 t1_jdy5tok wrote

> if chatgpt had memory, RAM, a network time clock, and a starting prompt, it would be sentient. So it already is.

I feel like you don't really understand how LLMs work. It's not me in a dark room, it literally doesn't do anything until you feed it a token. So there's nothing to be aware of, it's just a bunch of inert floating point numbers.

But even after you give it a token, it doesn't decide to say something. You basically get back a list of every predefined token with a probability associated with it. So that might just be a large array of 30k-60k floats.

At that point, there are various strategies for picking a token. You can just pick the one that has the highest value from the whole list, you can pick one of the top X items from the list randomly, etc. That part of it involves very simple functions that basically any developer could write without too much trouble.

Now, I'm not an expert but I do know a little more than the average person. I actually just got done implementing a simple one based on the RWKV approach rather than transformers:

The first line is the prompt, the rest is from a very small (430M parameter) model:

In a shocking finding, scientist discovered a herd of dragons living in a remote, previously unexplored valley, in Tibet. Even more surprising to the researchers was the fact that the dragons spoke perfect Chinese.

The creatures even fought with each other!

The Tibet researchers are calling the dragons “Manchurian Dragons” because of the overwhelming mass of skulls they found buried in a mountain somewhere in Tibet.

The team discovered that the dragon family is between 80 and 140 in number, of which a little over 50 will ever make it to the top.

Tibet was the home of the “Amitai Brahmans” (c. 3800 BC) until the arrival of Buddhism. These people are the ancestor of the Chinese and Tibetan people.

According to anthropologist John H. Lee, “The Tibetan languages share about a quarter of their vocabulary with the language of the Tibetan Buddhist priests.” [end of text]


flexaplext OP t1_jdxxwnv wrote

It's really quite a strange experience if you properly delve deep into your conscious thought process and think about exactly what's going on in there.

This subconscious supercomputer in the back of your mind that's always running, throwing ideas into your thought process, processing and analysing and prioritising every single input of this massive stream of sensory data, storing, retrieving memories, managing your heartbeat and internal body systems.

There's this computer back there doing so, so much on autopilot and you have no direct access to it or control over it.

The strangest thing of all, though, is this way it just throws ideas and concepts, words into your conscious dialog. Maybe I think that's strangest to me though, just because it's the only thing I am able to have a true perception of it doing.

Like I said, it's not necessarily single words that it is throwing at you, but overarching ideas. However, maybe these ideas are just like single word terms, like a macro, and then that single term is expanded out into multiple words based on the sequence of words in such a term.

There are different ways to test and manipulate its output to you though. You have some conscious control over its functionality. 

If you try to, you can tell and make your subconscious only throw out overarching ideas to you, rather than a string of words. Well, I can anyway.

You can also, like, force the output to slow down completely and force it to give you literally only one word at a time and not think at all about an overarching idea of the sentence. Again, I can do that anyway.

It's just like my thought process is completely slowed down and limited. It's just way more limited in thought and it's literally like the subconscious is just throwing one word at a time into my mind. I mean I can write out exactly what it comes up with when I do this:

"Hello, my name is something you should not come up with. How about your mom goes to prison. What's for tea tonight. I don't know how you're doing this but it's interesting. How come I'm so alone in the world. Where is the next tablet coming from."

I mean, fuck. That's weird to do. You should try it if you can. Just completely slow down and force your thoughts into completely singular words. Make sure to not let any ideas or concepts enter your mind. I mean, that output is way less than an LLMs capability when I do that, it's very, very similar to what basic predictive text currently is. In fact, it feels almost the same except that it appears to be affected by emotion and sensory input.

Edit: There is another way I can do it. Just think or even better speak out loud fairly fast without thinking at all about what you're saying. Don't give yourself time to think or for ideas to come into your mind. You wind up just stringing nonsensical words together. Sometimes there's a coherent sentence in there from where a concept pops in, but it's mainly still just like a random string of predictive text.


Frumpagumpus t1_jdyjjkk wrote

you have reasoned enough, it's time to go read source code and get something running

you can ask chatgpt to guide you