I understand that. I am pointing out the fact that they started on different paths. One of them was actually matching its name with what it was doing; the other was a contradiction from the beginning.

Edit: Wow, people either can't read or don't read enough history.

OpenAI has been open initially. From history books, it looks like RC was never P's.

I just meant this would take significant amount of time. I think it is impossible to do research in a quantitative field without understanding these so I would say it is well worth the investment. But most people are not concerned with research or even understanding the methods.

If you are serious, I would recommend working on Rudin's Principles of Math Analysis. It might take a day (or more...) to wrap your head around a single proof but at the end you'll be ready to read anything (of course you might need to check some definitions.)

For KL divergence, entropy etc., Info Theory book by Mackay is great.

For hessian, well it is just calculus; the second derivative of a multivariate function. To understand its uses, you would need some understanding of numerical analysis and concave programming. For the latter, Boyd's optimization book is a classic. I don't remember a good book on numerical analysis but some diff. eqn.s books have nice chapters on it.

Based on what you copied: they are saying that dropout introduces bias. Hence, it reduces the variance.

Here is why it might be bothering you: bias-variance trade-off makes sense if you are on the efficient frontier, ie cramer-rao bound should hold with equality for trade-off to make sense. You can always have a model with a higher bias AND a higher variance; introducing bias doesn't necessarily reduce the variance.

The thing is at ~$100/month, there are better and cheaper alternatives that give more option so it is hard to believe this is not an error.

I mean it is pretty cheap. You probably can't spend more than $10/month if it is priced similar to gpt3.