Reading comprehension and listening comprehension are two totally different things, and have to be built up independently.

When I first tried audiobooks, I couldn't follow them either (I'm not a native English speaker either). I had to work my way up from conversational podcasts --> history podcasts where I already know the broad strokes --> audiobooks of stuff like Pride & Prejudice, things I basically have memorized by heart at this point --> other podcasts where I don't know the material --> simpler audiobooks or books on topics I already know about (light sci-fi, greek mythology) --> more complex audiobooks. This whole process took about 3 years for me. Then there was another 18 months or so of listening to audiobooks at 1x speed (which is so much slower than my reading speed, but eh, I listened to audiobooks only in situations I couldn't read anyway - so something is better than nothing) before I increased my default listening speed to 1.5x (still slow it back to 1x or 1.2x for some narrators, and increase to 2x for others). But even after nearly half a dozen years, audiobooks still take me longer than reading the same book, and I am still much better at processing complexity in text than on audio (so I would always choose to read some books on text). Which is to be expected, I have been reading for much longer!

It feels like people just expect to pick up a complex book on audio and immediately listen at 2x speed. That's like a 13 year old picking up Tolstoy and expecting to get through it in a week. It's unrealistic for the average human being - you gotta start smaller!