I have a theory that the main reason OpenAI decided to start keeping it's training and architectural details private is because through minor modification in training data and data augmentation they were able to gain significant improvements in the qualitative output of GPT.

Thus any competitor could replicate the pipeline with ease and reproduce the improvements, so they decided to keep it as a trade secret.

Glad more research like this is being done and shared to the rest of the community.