Recurrent Parameter Generators https://arxiv.org/abs/2107.07110 could be interesting for you. The idea is not to prune the architecture, but instead use a limited bank of parameters that generates the networks parameters