Most likely you won’t need to code everything from scratch. You’ll probably just need to add an nn.Linear or a 1x1 conv to convert whatever dimension your brain activity data is into the dimension of the tensor that it is currently conditioned on (I think it’s 1024 or 2048 dim embeddings currently not exactly sure)