MrOfficialCandy
MrOfficialCandy OP t1_j7r7xg3 wrote
Reply to comment by uoftsuxalot in [D] Is English the optimal language to train NLP models on? by MrOfficialCandy
It's not just about swapping tokens for other tokens. It's that grammatical structure (of any language) which can convey ambiguous meaning.
MrOfficialCandy OP t1_j7r85li wrote
Reply to comment by like_a_tensor in [D] Is English the optimal language to train NLP models on? by MrOfficialCandy
That doesn't help at all. Reading tokens at the byte level does not stop the word "they" or "it" from being vague in the context of a sentence.