Twitter is already established as a brand to near saturation and Elon has more money than god. It's the perfect combo for ML philanthropy. Now waiting for that Tesla vision algorithm...

You'll probably be fine learning transformers directly, but a better understanding of RNNs might make some of the NLP tutorials/papers containing transformers more easily comprehensible.

Attention is an very important component of transformers, but attention can be applied to RNNs, too.

Yet people here praise Torch when Tensorflow equivalents are often faster in production. Tensorflow still has relevance and gets a bit too much hate here (and I personally prefer pytorch).

Yes, this is 'instance' or 'sample' weighting. You can choose to apply this weight to the loss or the gradients before your parameter update.

Trees complicate it a bit more. I've never done it for something like that, but check this instance weight input to xgboost as an example. In the xgboost fit function, there is an input for sample_weight.

I know that tensorflow has a new-ish library for trees. You could manually write a gradient descent loop with modified minibatch gradients there, potentially.

You could use a custom training loop where you down-weight the gradients of the unreliable samples before you do parameter updates.

Are the outputs of your model binary? You could instead set the target of your uncertain data points to somewhere closer to the middle instead of 0 and 1.

If you are training in batches, you could reduce the size of the gradient updates coming from the unreliable data.

Your videos have been great so far! Can't wait for more modeling content.

I'm referring to their new Make-A-Video model, but I suppose they just announced that a few days ago. Hopefully they fully release that model.

It seems that Google is being very conservative with the release of their diffusion models compared to even Meta and OpenAI's closed-source approach.

Luckily, Stability AI seems to be working on a video generating diffusion model.

I believe in figure 3.8, there seems to be a small typo.

> g-h) The clipped planes are then weighted

should be:

> g-i) The clipped planes are then weighted

Let me know if I'm mistaken here. Good stuff so far!

Edit: Added github issue regarding this.