It sounds like you just need straight compression. Not neural network compression but like storage of numbers compression. That’s going to be 10x or 100x better than whatever pruning will do for you. There is research in the field of compressing models for storage or transmission.