Except you are using a 2d example to represent 3d space. This is a common problem with trying to portray infinity in a way thats understandable to everyone. We, as matter, exist with a physical 3 dimension. The existence of matter allows us to determine a center as long as we can observe that matter in it's entirety. Thus is true whether or not we live within an infinite or finite space.

If space is finite, that center is defined by its edges.

If it's infinite, and assuming that the only matter within that universe came from finite number of big bangs, then its center would be defined by the distribution of matter within it.

If it's infinite, and assuming an infinite number of big bangs, then a center cannot be determined as there exists no definable edge or boundary of matter.

Going back to your ball example and why it is a bad representation of your argument, you could define a central point. First problem, the surface of a ball is finite. Ignoring that let's look at problem two. If you add any matter to the surface of that sphere, you now have a definable point in space. As it is a single point, it would become the definable center of that surface. As you add more points, the center would be defined by a point in space that would see an even distribution of points on the surface.