I don't really unserstand the question What do you mean "looking at a screen"? Or "looking at numbers and finding a pattern"?

The model takes in multidimensional array as input. That array is all the rgb values at a given instant. Take that to mean whatever suits you.