Consider Newtonian gravity. If an object falls directly into the gravitating body with no sideways motion, it will simply collide. It's orbital angular momentum that causes the object to be ejected back outward.

How does this work? If you write down the equation of motion for the orbital distance r, two forces emerge. One is the gravitational attraction, which scales as 1/r^(2). The other is a centrifugal repulsion term, which scales as L^(2)/r^(3), where L is the angular momentum. As the distance r becomes small, the centrifugal repulsion eventually dominates, ejecting the object back out to apocenter, as you say.

This works because the attractive force scales as 1/r^(2) at distance r. If the attractive force scaled as 1/r^(3) or steeper, then the centrifugal repulsion would be no longer guaranteed to overpower the gravitational attraction at sufficiently small radii, so there would be nothing to prevent the orbiting object from eventually colliding with the central gravitating body.

While gravity in general relativity can't be exactly described by a radial force law, the same basic idea applies. See for example how a number of 1/r^(3), 1/r^(4), etc. terms arise in the post-Newtonian expansion (scroll to equation 203).

That's true for nonrotating black holes, anyway. In the idealized rotating black hole solution, it is actually possible for the centrifugal repulsion to overpower the gravitational attraction! This is what leads to the crazy conformal diagram for a rotating black hole that suggests you can fall in and emerge back out in a different universe. However, there are many good reasons to expect that this idea does not work for realistic black holes and is just an artifact of the idealized construction.