p-morais t1_izvyzit wrote
Reply to comment by patient_zer00 in [D] - Has Open AI said what ChatGPT's architecture is? What technique is it using to "remember" previous prompts? by 029187
It’s instructGPT, which is based on GPT3.5 with RLHF. People have reversed engineered that it uses a context window of 8,192 tokens and primed with a special prompt.
rePAN6517 t1_izw4vqj wrote
Need a source for the 8192 context window. Last I heard it was 4000.
f10101 t1_izwhslb wrote
To confirm, 4k tokens is indeed what their FAQ says. https://help.openai.com/en/articles/6787051-does-chatgpt-remember-what-happened-earlier-in-the-conversation
sandboxsuperhero t1_izw2k3k wrote
Where did you see this? text-davinci-003 (which seems to be GOT3.5) has a context window of ~4000 tokens.
42gauge t1_izw3a1a wrote
What's the special prompt?
Viewing a single comment thread. View all comments