It's not hard to believe. They probably didn't have the right external conditions, like climate, which would need to be quite stable and warm to allow for large civilizations as a ubiquitous phenomenon.

They maybe did rarely have small rudimentary civilizations that could only prosper at trading nodes even tens of thousands of years ago, but it wasn't anything stable.

So hunter gatherers probably don't just all build civilizations anywhere on their own if enough time passes, because they first need the right conditions in order to prosper. Moving rocks requires a lot of people which would need to be fed. This requires large and stable food supplies i.e. a stable climate. If the climate changes all that collapses.

And because those civilizations were rare, they could hardly build off of another before all their knowledge was lost. It took a few hundreds of thousands of years plus the right kind of global climate for that effect to compound to the level where it is now. So, it wasn't because humans back then were inherently stupid.