Expansion of the Universe aside, the size of the observable Universe (the only "size of the Universe" we can really observe) is approximately the age of the Universe times the speed of light. (Expansion has increased it well beyond this value - it's about four times that - but it's still on roughly the same order of magnitude.) So a simple answer to your question is "the Universe is old, so light has had a long time to travel, so the most distant parts we can see are very far away".