Well, it's infinite. Always been. So it's just expanding. Not "into" anything.

It might help to think of it as "the universe is thinning out".

The thing is, the universe was in a state of expansion from day one. Pretty violent expansion. You could call it an explosion. Or a "big bang". It's been expanding / exploding ever since.