Your claim that prompting can achieve what fine tuning can't contradicts the documentation for openai that you posted that said fine tuning can do whatever prompting can without the length limit

Huh? Have you never included text in a prompt and asked it to answer questions about the text? Seems like that counts as "new knowledge" by your definition

Looks like this was trained on just 150 x-rays and does very well: https://paperswithcode.com/paper/xnet-a-convolutional-neural-network-cnn

Edit: did you look for pre-existing solutions? This was like the second google result. If I were you I'd be looking for public datasets I could use for pretraining and then finetune on my data

The second link says fine tuning is a substitute for lengthy prompts, including putting more into it than can fit in the longest prompt. Prompts are a way to give the model new information. What is your definition of knowledge that isn't something you can put into a prompt?

I'm just talking about diffusion models in general and the concept of denoising. LLMs are what you would use, not the way you'd train a diffusion model but the way you'd train an LLM

Compilation isn't a process of noising and diffusion doesn't have any relevance here. An LLM is what you would use

In my understanding, if you have text, it's not a challenge to train on next word prediction. Just keep the learning rate low. The reason there's a focus on the instruction based fine tuning is because that data is harder to come by.

My only experience is I've done this with a sentence embedding model (using sbert) and I just trained on my new text and the original training data 50/50 and it both got better at embedding my text and didn't forget how to do what it was originally trained on

Keep the laptop, buy a used gaming desktop with a 3060 12GB VRAM and ssh into it from your laptop.

The source I found this post through also referenced Retrieval Augmented Generation (https://ai.facebook.com/blog/retrieval-augmented-generation-streamlining-the-creation-of-intelligent-natural-language-processing-models/) and it seems like they've integrated document selection into the back propagation of the model training. You couldn't do this with chat GPT but maybe smaller pretrained LLM that could be fine tuned on consumer hardware would be enough for just that part

Yeah I brought a used gaming desktop from Facebook and kept my 6 year old laptop. Crazy specs for the price and came with a 3060 with 12gb vram. I recommend a gpu with more vram vs one that's "faster" because it won't be fast if it can't load the model at all

Huh?

idx = unravel_indices(indices, shape) Values=arr[*idx]

No loop required. If you're referring to the same loop you were using to get the argmax, you can just adjust your indices first so they apply to the unstrided array

I think unravel is a tuple so you can just star unpack it to use it as indices without having to do anything else with it

I'm away from a computer for a while but you could cast the tuple to an array I assume. And since creating an array is expensive and you'll keep needing an array of the same shape every step, you could just hold onto it and assign values into it instead of re-creating it every time

Yeah, I was just thinking in 1D. Im not at a computer so I can't try anything but roughly what I'm thinking is you have a (H, W, D) array and use stride tricks to get a (H, W, D, wx, wy). If you could get that to be (H, W, D, wx*wy) then argmax could give you a (H, W, D) array of indices. I dunno if you can reshape a strided array or use strides to get the shape in question

And we'd all be finishing each other's... sandwiches?

Can't you just reshape the array and use argmax (so no as_strided). Reshaping is often free. You'd have to do some arithmetic to get the indices for the original shape, but it would just be one operation

I.e. you can take a shape (99,) array and reshape it to (3,33) and then get 33 maxes.

It was a joke

I assume the program is installed into the virtual environment and so is operating within it. That would be done with the console_scripts entry point

It was trained on python 2

No I don't have time for that. Good luck

Latent diffusion works with text because Clip was trained on millions of pairs of text and image already. You've got a huge project of training millions of brain activity/text pairs ahead of you

Maybe your learning rate is way too high. Is this tensor flow or something? Or are you writing this from scratch?

The fact that you can throw a bunch of compute at transformers is part of their superiority. Even if it's the only factor, its really important

Neither. Just plan to ssh into an aws instance if you're going to have a laptop. I bought a used gaming desktop with a 3060 (12GB vram) and otherwise great specs for like 900 bucks and I ssh into it from my 6 year old laptop.