Viewing a single comment thread. View all comments

soraki_soladead t1_izjpxd6 wrote

Reply to comment by jcasper in [R] Large language models are not zero-shot communicators by mrx-ai

fwiw, it is very difficult to know if the model has seen the task (or similar tasks) before or not due to the nature of the data collection

I feel like zero shot / few shot has taken a much less rigorous tone when applied to LLMs

15

FutureIsMine t1_izkmv5n wrote

hmmmmm, to some extent I think you might be onto something, there could be something related, on the other hand, seeing enough data means that you've seen bits and pieces of a zero-shot task so its not exactly "seen" but its not brand new and a novel task, its just piecing it together from multiple other tasks

2