Specialist-Regret241 t1_izfecx7 wrote
Well done on Cicero - I played against it three times in August and the only odd thing about it was that didn't engage in the post-game discussion.
Question - how do you think Cicero would fare with more time for discussion? I don't tend to play games with turns that are less than 2 days, and blitz only has 5 minute turns. Or is that something you can't easily test now that the active population of blitz players knows about Cicero? I for one will no longer assume I'm playing against a human when I use webdip in the future.
MetaAI_Official OP t1_izfmiav wrote
As noted in an answer to a previous question: we were originally targeting 24hr-turn games, but ended up pivoting to 5min-turn games due to the inability to gather a sufficient number of samples in the 24hr-turn format (as playing a single game can sometimes take months)! Playing 24hr-turn games would indeed pose additional challenges from a language generation perspective — while human players tend to send a similar number of messages in each format, messages in 24hr turns tend to be significantly longer (and likely more complex). Moreover, human players would have more time to interrogate mistakes from the bot, which could potentially lead to the agent making further mistakes. -ED
MetaAI_Official OP t1_izfq7yw wrote
Regarding the post-game kibbitzing, we discussed this a few times, but every solution felt like we'd be faking it. For example, we could have put a human in the loop here but.... why? In the end we picked the most honest approach we could when dealing with the community, which was an ethical consideration that underpinned the whole project I think. -AG
Viewing a single comment thread. View all comments