Essentially, you argue that reading (fiction) is self-improvement.

Is it true? Can you measure such improvement somehow? If you can't, is it meaningful?

Regardless of whether it is true, if your goal is persuasion, I don't think this argument will work.

It puts you in the rhetorical position of knowing better than they do, because you've exercised your mind and they haven't. This is only likely to make your interlocutor more defensive and disparaging of your "book-learning".

People who disparage "book-learning" generally do so out of insecurity, because they don't have any. You can't win an "argument" with such a person by making them more insecure.