High-speed cameras and deep learning [Research]

Submitted by A15L t3_10v12fn in MachineLearning

I haven’t been able to find research on deep learning using high-speed cameras that capture images at frame rates higher than 250fps. I wonder if they are rather useless for image/video processing or do any of you have any ideas about potential applications.

2

Comments

You must log in or register to comment.

ricafernandes t1_j7f1t9v wrote

The thing is probably processing time. Not many good image models or applications can run 250 times a second in order to process each of these frames, as they usually have longer processing time than some ms

0

PassionatePossum t1_j7f1xr8 wrote

I'm not sure I follow the question. Why would there need to be special research for high-FPS cameras? The challenge with all video-based systems is to capture long-range dependencies. And "long-range" is defined over the number of frames. How much time has elapsed between the frames doesn't really matter.

However, if you have a high-FPS camera and a slow moving scene, you'll have a lot of images are are pretty much identical to each other. That means, according to information theory there is very little additional information in each frame. In that case you might want to consider to do temporal downsampling on your data. If you have a fast moving scene and you really need to take advantage of updating your prediction for every single frame, the only constraint is processing power.

So in that case, the problem of inference for high-FPS cameras is the same as computationally efficient models. And there are a few models who are intended to be run on mobile devices. Maybe you want to look into that.

4

_d0s_ t1_j7g250y wrote

Not the same, but I'd suggest to look into event cameras

2

noone_relevant t1_j7gamsz wrote

You said it yourself in the question. What is the potential application? Also does it have to be online? If not even though the camera is high speed it is similar to other camera for deep learning

1

bernhard-lehner t1_j7jgi4y wrote

One practical issue with high speed cameras is the lightning that is required to still get enough exposure. Depending on your situation, you might draw in a lot of bugs, which can then negatively interfere with your system.

1