[D] What happens if you give as input to bard or GPT4 an ASCII version of a screenshot of a video game and ask it from what game it has been taken or to describe the next likely action or the input?

Submitted by Periplokos t3_121nwoi in MachineLearning

​

I was wondering of what would happen in the following scenario which I can't try since Bard is not supported in my country and I do not have GPT4:

>What happens if you give as input to bard or GPT4 an ASCII version of a screenshot of a video game and ask it from what game it has been taken or to describe the next likely action or the input?

0

Comments

You must log in or register to comment.

Fit-Recognition9795 t1_jdmsxa9 wrote

As an AI language model, both GPT-4 and its predecessors like me, ChatGPT, are designed to process and generate text, not to analyze images or visual data. Giving an ASCII representation of a screenshot to GPT-4 or any text-based language model would likely result in a poor understanding of the actual image, as the model doesn't have the capability to process images in the same way that a human or a dedicated image recognition AI can.

However, if the ASCII representation is clear enough and contains easily recognizable elements that are unique to a particular video game, there is a chance that GPT-4 might be able to make an educated guess about the game in question, but the accuracy would be significantly lower compared to proper image recognition AI.

Regarding the prediction of the next likely action or input, GPT-4 might be able to provide some generic suggestions based on the text description, but again, its ability to understand the actual visual information would be limited.

For analyzing images and making predictions about visual content, you would be better off using a dedicated image recognition AI model, such as OpenAI's DALL-E or an AI model specifically trained for video game analysis.

8

Eggy-Toast t1_jdn7cki wrote

ChatGPT doesn’t know that GPT-4 has multimodal input though, right? I assume based on “not [designed] to analyze images or visual data” this is the case.

2

Fit-Recognition9795 t1_jdnbjar wrote

Even worse, gpt-4 doesn't know it is gpt-4.

I have chatgpt-plus and the above answer is generated using the gpt-4 model.

2

SeymourBits t1_jdnttwn wrote

Interesting experiment. I have not done it but I predict it would hallucinate a well-documented video game screen, like Pac-man, and then describe probable actions within the hallucinated game.

2

Difficult_Bid_9828 t1_jdom9pj wrote

I tried something like this by transforming a random image of a game I found (Doom and Mario 64) into ASCII art and giving it to Bing with the instruction to guess which game it was from. Did not work.

2

mskogly t1_jdq58o4 wrote

I believe gpt4 can read and describe the content of images. No need to go via ascii.

1