I agree with everyone else here. For starters, while it's okay to not read novels (or, really, engage in ANYTHING that you don't want to) for your own reasons, the reasons you state don't seem the most healthy.

Which, hey, I get; I get upset with myself when I can't get through puzzles quickly or get gold on a challenge run or whatever, but that is decidedly a *me* issue rather than an issue with the product or medium itself.

If it makes you feel that bad, then don't worry about forcing yourself through it. Your life is your own, and you owe it to nobody to live it a certain way. But it may behoove you to introspect and find a deeper meaning as to why it upsets you so much, as this kind of feeling could stop you from taking enjoyment in a lot of other parts of your life too.