Exactly. I mean I can easily define L2-constraints for the weights of my network and then do constrained optimization, which would at least theoretically be equivalent to L2-regularization/weight decay. But this is not quite useful, I am wondering whether there are applications of constraints where it actually makes sense.