Exactly. I mean I can easily define L2-constraints for the weights of my network and then do constrained optimization, which would at least theoretically be equivalent to L2-regularization/weight decay. But this is not quite useful, I am wondering whether there are applications of constraints where it actually makes sense.
d0cmorris OP t1_j819chm wrote
Reply to comment by tdgros in [D] Constrained Optimization in Deep Learning by d0cmorris
Exactly. I mean I can easily define L2-constraints for the weights of my network and then do constrained optimization, which would at least theoretically be equivalent to L2-regularization/weight decay. But this is not quite useful, I am wondering whether there are applications of constraints where it actually makes sense.