Unify transformer_lm_megatron and transformer_lm (!633) · Merge requests · Administrator / metaseq

Closed Administrator requested to merge github/fork/bashnick/Unify-transformer_lm_megatron-and-transformer_lm-arch into main Feb 03, 2023

Created by: bashnick

Patch Description

Remove *_mp.py files
Unify transformer_lm_megatron and transformer_lm architectures

Testing steps python -m METASEQ_.projects.PROJECT_NAME.sweep_baseline -g 4 -n 1 --rsc --model-size 8m --tokenizer rsc --prefix $RN --local --data /checkpoint/TEAM_NAME/datasets/consolidated/v4.0