The following work shows that with sufficiently large width (overparameterized regime) you can have polynomial convergence to the global minimum which gets worse (but polynomially) with the depth of the network.

A Convergence Theory for Deep Learning via Over-Parameterization