> I think the obvious truth that we generally don't choose our preferences is inherently problematic for the common concept of free will.

But why? First, this has little to nothing to do with determinism. It's also not like humans have spent thousands of years under some illusion that they choose their preferences, since as you point out it's quite obvious. It's clear that people don't generally think this poses a problem for free will or moral responsibility. So why do you?

> Can you explain what makes you think your definition is sensible?

It's a necessary but not sufficient condition, and wasn't intended to be a total definition. It's not easily possible to provide an all-encompaasing definition of something so nebulous as morality that'll get wide appeal.

> no way to distinguish between the processes of the brain which give rise to mental states, including thoughts and choices, and simpler deterministic mechanical systems like internal combustion engines or computers

What does this have to do with determinism? Isn't this just the hard problem of consciousness? It's just as hard for me to look at a brain and find the processes that give rise to mental states in a dualistic universe.

> least until we developed computer programs sophisticated enough to trick people's pattern recognition algorithms into interpreting stimulus from a computer as stimulus from a mind. But even where that trick is effective,

It feels a bit like having your cake and eating it too when you jump from saying that people's intuitions matter when they judge an internal combustion engine as not making choices but somehow don't matter if they were to judge a sophisticated computer algorithm as making them. In any case this:

> that's a pretty convincing argument to most people that chatGPT isn't actually a mind.

Is an empirical statement. Do you have something to back it up? Specifically that people don't think chatGPT has a mind because it's output is predetermined by mechanical processes. Also, what do you think of the fact that people commonly apply "choose"-type verbiage the output of things like recommendation and search algorithms (e.g. "Let's see what YouTube picked for me today")?

> Most people conceive of free will as existing in the universe where there is a possible counterfactual to a choice.

In the specific sense of "possible counterfactual" you go on to elaborate? I don't agree, and this contradicts what I know of the current results in experimental philosophy. What makes you think this? Especially when you go on to try to say that mass amounts of everyday communication are actually nonsensical.

> the word "choose" only makes sense if there is a possible world in which I ate cauliflower

If your definition of a word implies that it's constantly misused, your definition is just wrong. People say things like your example all the time and clearly understand each other. If you're saying that their use of the word doesn't make sense, you've effectively shown that your definition has nothing to do with they way the word actually gets used.

> If there was never a possibility that I would "choose" cauliflower, I didn't make a choice.

In the context of our discussion, this is question begging. The essence of the compatibilist position is that you make choices even when there wasn't any possibility you would choose differently.

> My mental processes didn't have any effect on the outcome.

They quite obviously did, since they're part of the causal chain that resulted in the broccoli eating. Without them no broccoli gets eaten. They're essential to and directly caused the outcome you're trying to say they didn't effect. This is like trying to say that one ball colliding with another didn't have any effect on the second, now moving, ball.

> In other words, a choice is the ability to actually change the future state of the universe via internal mental processes.

But you have that. Your mental processes absolutely do determine the future state of the universe because they, obviously, cause things to happen. You're trying to also say that it's somehow problematic that those mental processes themselves could not have been otherwise and are themselves the effect of another cause, but it's still not clear to me why you think that.

> There is no "me"

I have absolutely no idea what determinism is supposed to have to do with establishing a line between self and not-self.

> The only time it is possible for free will to exist is if my mental processes are not entirely predetermined the history of the universe up to the current point. Only that allows me to change the pattern of activation of neurons in my brain and central nervous system and muscles so that I can effectuate my genuine preference. Otherwise my body is a mechanism and everything that happens in the mechanism is fully automatic.

But how does this help? This touches on the original question I asked you: what other factor are you requiring in a decision for it to qualify as free? On what basis would you decide otherwise or change your neural firing? It can't be anything about who you are or the facts of the situation, so why would you ever decide otherwise? It certainly can't be your preferences, because those already exist when a decision comes round and quite obviously affect your choices in a deterministic universe.

> But in the sense that people commonly understand it, it also requires that the universe not predetermine our choices

This is another empirical claim. What evidence makes you believe it?