quagg_
quagg_ t1_izn5ha7 wrote
Reply to comment by Mefaso in [D] When to use 1x1 convolution by Ananth_A_007
No 3rd party ones (that I know of), but I've an implementation of my own. Give me a day and I'll setup a repo to share it!
quagg_ t1_izjvgkv wrote
Reply to comment by MathChief in [D] When to use 1x1 convolution by Ananth_A_007
To add onto this point about it being the same as nn.Linear in PyTorch, it is useful in hypernetwork applications where you have a data-conditioned (a context set, partially observed sequence, etc.) hypernetwork. Because of the data-conditioning, each sample has a different main-network MLP and doesn't inherently allow for batching.
If you want to parallelize over multiple different MLPs at one time still, then you can use 1x1 convolutions in 1D alongside the "groups" tag to enable running through all of those separate networks at the same time, saving you from sequential processing at the cost of a larger CNN (NumNodes * BatchSize as filters) in each convolution layer.
quagg_ t1_izr231j wrote
Reply to comment by Mefaso in [D] When to use 1x1 convolution by Ananth_A_007
Here's the first (quick) version up! https://github.com/qu-gg/torch-hypernetwork-tutorials
It currently only contains simple examples on MNIST to highlight the implementation structure, however I'll add time-series (e.g. neural ODE) examples and better README explanations when I can over time. Let me know if you have any questions or feel free to just ask in the Issues.
Thanks!