[P] 🔥 CleanRL has reached v1.0.0; Reworked documentation, JAX support, and more!

Submitted by vwxyzjn t3_yvjm1r in MachineLearning

🔥 CleanRL's paper has been accepted to the Journal of Machine Learning Research! Introducing CleanRL at v1.0.0, a DRL library that is easy to read, well-documented, and adequately benchmarked. We have added reworked documentation, JAX support, hyperparameter tuning, and more.

Here are the links:

  1. GitHub Release: https://github.com/vwxyzjn/cleanrl/releases/tag/v1.0.0
  2. Twitter thread: https://twitter.com/vwxyzjn/status/1592246430043103232
  3. Paper Link: https://jmlr.org/papers/v23/21-1342.html

I am the main author & feel free to ask me anything here.

70

Comments

You must log in or register to comment.

VinnyVeritas t1_iwfnzp9 wrote

This is amazing, the code is so clear I feel I can finally understand RL!

Bookmarked.

7

sidhire t1_iwfrsr0 wrote

Nice project! FYI the PPO docs page keeps crashing for me.

5

vwxyzjn OP t1_iwgdq6v wrote

Hey, thank you! Sorry to hear that. How does it crash? I noticed on the phone the PPO docs kind of crashed because the page was too long.

2

SatoshiNotMe t1_iwgk7ht wrote

Great work, and congrats!

If I may make a suggestion for future. Suppose I invested months in another RL framework, say SB3 along with its accompanying zoo (experiment manager, trainer etc), plus an elaborate HP tuning (e.g. with RayTune) code. It would be great if you can provide a recipe for how to surgically alter this type of sb3-based code to use CleanRL's model-implementation instead. Granted the recipe cannot address all needs, but even an example would be valuable. Essentially a type of "migration guide" from other frameworks.

Thanks !

2

snorglus t1_iwgd1uu wrote

Fyi: Many of the links seem to be broken.

1

vwxyzjn OP t1_iwgdstt wrote

Oh whoops I messed up the links in the github release. Should be good now.

1

girvanabhasarasasvad t1_iwiipzy wrote

This is a boon for people new to RL, congrats! After looking at the implementations, I am curious whether you have heard of SaLinA? It's a lightweight library for RL (and sequential learning in general), with a design philosophy that's very similar to CleanRL.

1

RSchaeffer t1_iwiq0gq wrote

Where's a good place to learn about the landscape of different RL libraries and understand how CleanRL compares to them?

1

carlml t1_iwf9k70 wrote

why did you create it among all the other things you could have done?

−16

Toilet2000 t1_iwg9fsl wrote

Why did you comment this among all the other things you could have commented?

6

carlml t1_iwgu3gb wrote

because I want to know the reason why he created it. I also want to know if it will be maintained for a while.

1

Toilet2000 t1_iwgwg4j wrote

Then your comment is very badly worded. It sounds a lot more like you’re telling them they could’ve done something else.

6