> Writing a description of every step instead of just clicking seems like a downgrade to me.

Use a LLM to write the step by step prompts as well. Like SayCan

> We show how low-level tasks can be combined with large language models so that the language model provides high-level knowledge about the procedures for performing complex and temporally extended instructions, while value functions associated with these tasks provide the grounding necessary to connect this knowledge to a particular physical environment.