Quite a few well-funded projects going on right now for 3d printing modular construction materials as well. AI is targeting information work because it is the cheapest to train on and implement into products. See self-driving cars for where there is overlap, but physical work is still too expensive to be the leading force of AI development. Soon though...

Language is an encoding scheme of our intelligence. It is enough to model our intelligence, I'm sure. But I don't think it is enough to build an intelligent agent. The agency that humans have I think is old and not rooted in our intelligence, rather it uses our intelligence. It's a carefully tuned array of interconnected processes in equilibrium that respond to disturbances in our environment, all encoded by our genetics. I suspect that part will be much harder to get right, as the nuance of building an agent like a biological social animal for example, is no doubt tremendous. Evolution has had a long time using trial and error to work out the issues. This is the part that unfortunately has the most potential to go terribly wrong.

Unless there's some news, I missed I don't think we've measured Hawking Radiation (certainly not from a black hole). However, it is well defined mathematically and appears independently through several different approaches. I don't think anyone credible is disputing hawking radiation regardless.

New from the makers of Second Life, introducing Final Life. VR finally brings you every experience possibly in reality, all from the comfort of your living pod. If you die in game you die in real life! ^(Requires neurotoxin content pack for full experience; sold separately.)

The word itself best colors what it is. The singularity is a point past which we know nothing about. A point where such immense potential for rapid change is possible, that what exactly happens will depend on a butterfly's wings.

The ramp up to the point though, where we commodify near-agi agents, will likely create many infantilized doofuses as you put it.

Do you acknowledge the legitimacy of subjective experiences of other humans as much as your own? Do you acknowledge the same for other species of intelligent animals? Of a plant? What of the individual cells within your body? Do they have subjective experiences? I don't know how anyone can acknowledge that and then consider themselves an atomic consciousness. If you do, where reasonably do you draw the line? I believe you are raising human subjective experience above all others by claiming any human experience is essential. An objective reality can co-exist with a subjective experience of it.

We both agree the red light is red, and that it exists. We can agree because the relation its properties have with our entire experience of the physical world is consistent. I can find some light and measure the wavelength using a photosensor, without ever laying eyes on it, and tell you it will be red. You will observe it with your own eyes and agree it is red. Our experience of what red actually looks like is entirely our own and in no way can be compared. It cannot even be considered that they might be different because the operation of comparison is simply undefined for subjective experience. If you claim objective reality doesn't exist, you are entertaining solipsism. I will maintain that all of our subjective experience is rooted in an objective reality. If you claim you've seen a new color, we can recreate this and explain what causes this. If it's not something simple like a wavelength of light only you can see, it might be something more nuanced, like an internal experience unique to the micro-structure of your physical brain.

Many people entertain the idea that quantum mechanics hints at a link between the subjective experience and the physical universe through various interpretations. I don't think lay-people (me included) should be adopting any of these interpretations as philosophical evidence. We simply don't know enough yet. There are many experiments that have been and will continue to be done that further clarify the root of them that is the measurement problem. Decoherence theories for example are helping to explain what appears to be "wavefunction collapse" as the absorption of the quantum properties by a larger quantum system.

A question to end this long reply, since you mentioned it in your original post: do you consider panpsychism to be a valid idea?

Agreed as well. What comes out the other side of this will probably not resemble original humans at all, but regardless, whatever intelligent agents are digitized will have a huge advantage over biological agents in nearly all aspects. Be it artificial or some kind of derivative of human flesh scans or individual behavioral modeling.

I suppose that is reasonable at least on a basic level; an appeal to probability. I can generally appreciate ideas that remove the requirement for us to be in a special/improbable position.

The original discussion was about what could be done with VR that could not be done in reality, but I digressed and commented on the more philosophical reasoning you had about why reality was better than VR.

To respond to this, I'll start by saying I'd be more careful about assuming so much about another person merely through a short text comment. My predictions come from my own small experience watching the world, as do yours. My philosophy is more akin to pragmatism, not nihilism.

Truely the bulk of my response was directed at your last remark, about assigning meaning to a specific type of experience and discounting others. An honest question: what do you propose "meaningfulness" is measuring in human experience? What do you propose quantifies this void you speak of?

>We most likely live in simulation

This has always been a line of thinking with no practical use. Even if the current laws are just some abstract rulesets layered on top a real physical world (like code in a computer), you're still physically real in the world that's running your simulation, just maybe not in the form you thought you were. But without simulation theory, we are certainly not perceiving the true depth of what we physically are. The only point it generates that seems relevant is that we are in a "false" world. But how would we even know?

To me it just adds one or more arbitrary layer of complexity without producing any philosophical value.

Simply put, VR will always be more adaptable and cheaper. The goal will not be to unlock a utopian world, but rather to supply the demand for escapism more and more human's desire. This is a physics problem first and foremost. It requires much less energy to represent an experience than to create it physically, as our brains necessarily much smaller in scope than the world they operate in. You can see a mountain and take that experience with you in your mind without having to move the mountain physically.

Maybe it's the cynic in me, but brains and utopia are incompatible. The brain navigates problems. Without the wonders of modern civilization, there is always a problem to solve. In our world, we sometimes don't have anything to do, as the systems around us do so much for us. So, we fill the gap with anything we can find.
"Diversions" as you say, even the "meaningful experiences with people" are this. There are diversions that speak to our more basic drives (start a family for example) that can sustain us for much longer than our artificial entertainment, but many minds "see through this" and find themselves on the losing side of a battle with their mental health. This is not an anomaly, it's the expected outcome of making the brain unemployable. In your utopia, if humans could live healthy for 1000 years, I suspect mental health disorders would be the leading cause of death.

Most humans live their lives completely disconnected from the rails of progress. Most don't need a grand ambition to be satisfied, instead they get by with rewarding distractions that keep their minds from collapsing inwards.

Remember the brain evolved to serve the body, not the other way around. Happy body, happy brain. I'm sure the AI will figure out how to pacify the population should that be its goal. Humans existed happily for at least 2 million years - most of which had almost no scientific progress, it's not required for a human mind. An ASI could easily revert human society to this sort of simplicity as the "ultimate" solution.

We are a long way away from these things if they ever happen, but there is an interesting convergence in technology that might happen in the long future. Basically, nano tech already exists, but we did not make it. Cellular life checks all the boxes nanotech aims to achieve. It is highly likely that if nanotech ever enters mainstream usage (consumer products, building materials, etc.) it will take the form of bio-engineered cellular life, not tiny robots. Why make your own when you can just hijack the existing proven platform? It's not likely this would take the form of large-scale plants, rather in microscopic things like fabrics, adhesives, maybe even data processors (so called wetware computers). Though it might happen that we use cybernetic engineered "animals" with remote control tech embedded into them for certain labor/exploratory tasks as self-maintaining drones.

IMO it's far more likely we achieve near perfect mastery of biology before we ever have serious civilization on other planets, even within our own solar system. Once we have that much control, it's just another form of machinery/tech.