Yep thats exactly my thought process. Science on drugs is often rare and not thorough enough to make definite conclusions but it's fair to say that whatever impairs REM sleep must impair memory and possibly other systems as well.