If you say you had a good understanding until then, what changed? The GPT architecture as far as I know in newer editions didn’t change completely, but made smaller changes and spent a lot of time on better data, better curation/guidelines etc.

Look up Andrej karpathys YouTube videos of building makemore from scratch

[removed]

he also had videos on building mini chat-gpt, man's a treasure

Vote for Midjourney. I don't know how they improved their performance, no paper or publications.

There was some discovery made recently about something to do with offset noise during training - people are speculating that MJ did that while others didn't. Here is video explaining how it works: https://m.youtube.com/watch?v=cVxQmbf3q7Q

On the other hand if that was it MJ would be better at generating dark images, so maybe not? Shame they don't share how they do it.

Thanks for the link, this methods sounds workable. The earilest version of MJ results was somehow blurry and noisey, I wonder if it was because of this method.

Regarding ChatGPT, I believe OP is frustrated not by the Transformer architecture but by the improvements made in the inference functionality.

That’s the real “black box” of GPT style LLMs and the least open

Just about every discussion i get notifications for is deleted , what s up with that?