Viewing a single comment thread. View all comments

Nameless1995 t1_j05chgd wrote

> If I repeatedly make the same request in the same thread, these characteristics of the responses do display more diversity, but the responses all have the same structure (e.g. the same number of paragraphs, and often near-identical sentences in corresponding paragraphs).

For proper results you should resample by clicking the "try again" button (or thread reset). Otherwise if by chance the first sample talks about a woman scientist named Samantha, all the later responses would be biased by that. Your next samples won't be independent by selectively biased based on the initial sample. To control for that, when comparing multiple samples you should make sure they are sampled under the similar conditions besides differences in rng (i.e use the "try again" given the same past conversation, or ask all of them in reset state).

> So I tried a simpler request, of giving me the name of a vegetable. I asked 35 times, and it said "carrot" 30 times and "broccoli" 5 times. The results of all my vegetable-name interactions are here. I also tried asking it to name an American president in 6 threads, and it said "George Washington" each time, and I tried asking it to name an intelligent person, and it usually said Albert Einstein, although it did occasionally say Stephen Hawking.

Sounds about expected.

1

Osemwaro OP t1_j06rbf6 wrote

I know -- all of the statistics that I gave are based on samples created in new threads or with "try again". I only mentioned what happens when I repeat a request within one thread to prove that ChatGPT knows the names of other vegetables, etc.

1