Submitted by enryu42 t3_122ppu0 in MachineLearning
farmingvillein t1_jdspflq wrote
Reply to comment by enryu42 in [D] GPT4 and coding problems by enryu42
So, don't know whether this actually makes a difference, but I'd review the overall post-conversion text.
E.g.: https://github.com/enryu43/llm_coding/blob/main/atcoder_eval/statements/statement_abc293_b.txt
You'll see that it represent "K" and "N" wrong here (in sample 1, 15 versus 5, 12 versus 2).
Certainly, as a human, I would find this confusing. Maybe you could get some automated robustness by telling it how you converted the text (as it might automatically adjust its "expectations" on interpreting the numbers). Obviously, the fairer comparison though would just be to fix this.
> as they require coming up with some ideas before writing the code.
The other thing I'd note--
Not sure whether you're using the API directly, but if I play around with these in ChatGPT, I often run into the context window and have to nurse it along to complete text. I'd make sure that however you're running things, you're giving it enough "space" to iterate (particularly if you use any reflection techniques).
Viewing a single comment thread. View all comments