This is a boon for people new to RL, congrats! After looking at the implementations, I am curious whether you have heard of SaLinA? It's a lightweight library for RL (and sequential learning in general), with a design philosophy that's very similar to CleanRL.