Yes I was implicitly talking about the final activation function. With regards to softmax, he said in another comment "I have a quantity of 100 units that need to be allocated across 50 days.", so I took that to imply the outputs should be positive (hence the exponential is reasonable).

I'm not sure I understand what you mean?

My first thought is to predict the 50 numbers simultaneously, and to apply softmax to the output (enforcing summing to 1), then scaling that so it sums to your desired number for each group.

The network is already doing its best at minimising the distance. If your final goal is point estimates that minimise the distance, predicting the error is probably not a good way to go about improving performance.

However, if you care about the uncertainty / having a distribution over where the ground truth might be, then there are definitely various techniques that allow this.

For example, if you expect the errors to change depending on some conditioning variable, you could have the neural network output the locations (mean), and the standard deviations (uncertainty) of the positions, given the conditioning variables. In practice you would output log stds and exponentiate to ensure positivity. Then you could use a Gaussian likelihood function approximation, replacing the L2 norm with the negative log likelihood under the Gaussian assumption.