Mostly that’s true. But given the right background, and a significant amount of luck, very small founder populations can expand enormously. This is very common in invasive species. The 200 million starlings in North America today arose from a few dozen in the late 1800s. Bull trout in Montana arose from two founders. A million Barbary Ground Squirrels arose from a single female.

Again, this is possible but not inevitable. Most such introductions will collapse due to inbreeding. But occasionally explosions can happen.