The observable universe is not infinite. And it keeps expanding such that more and more of it is not visible. But that's just because we're limited by time and the speed of light with respect to what we can see. If you went to the edge of the earth's observable universe you'd probably just find more universe. You'd be at the centre of a new observable bubble with the earth and a portion of the earth's observable universe bubble in one corner. As you move anywhere the "observable universe" moves with you. You take a step left and now the observable universe is one step larger to the left (though it's expanding faster than that so you wouldn't see anything new). Make sense?

It's not exactly like raisins in a loaf, though that's a good way to explain it to layperson. Unlike a loaf of bread or surface of a balloon, there isn't a firmament in which galaxies are suspended, and only the firmament expands such that they never touch. It's more like an infinite stew that keeps getting infinitely watered down - slowly. Sometimes the bits in the stew will still hit each other as everything cooks and swirls. They'll just do it less and less often as more water gets added.