[R] You Only Segment Once: Towards Real-Time Panoptic Segmentation [CVPR 2023]

Submitted by Technical-Vast1314 t3_125eo4f in MachineLearning

Happy to introduce our latest work on Panoptic Segmentation: YOSO. And to the best of our knowledge, YOSO is the first read-time panoptic segmentation framework that delivers competitive performance compared to the state-of-the-art models.

The code is available here: https://github.com/hujiecpp/YOSO

Specifically, YOSO achieves 46.4 PQ, 45.5 FPS on COCO; 52.5 PQ, 22.6 FPS on Cityscapes; 38.0 PQ, 35.4 FPS on ADE20k and 34.1 PQ 7.1 FPS on Mapillary Vistas.

​

https://preview.redd.it/5bmcl1n6bmqa1.png?width=1362&format=png&auto=webp&v=enabled&s=2f321fa31d8c3de5f53dbd61905a6448e9cb0a24

https://preview.redd.it/o4pkvri7bmqa1.png?width=674&format=png&auto=webp&v=enabled&s=fb609576c7ed615c3b0cb38a3db243eebe95e8e1

https://preview.redd.it/ufdaa028bmqa1.png?width=681&format=png&auto=webp&v=enabled&s=ecbceab2cb70ed8d63e4b37e7caa8068e63d367e

https://preview.redd.it/y2t2upj8bmqa1.png?width=603&format=png&auto=webp&v=enabled&s=4830fdb9817081466e2efad184ceb8c12b4b4f60

https://preview.redd.it/3wiyep09bmqa1.png?width=597&format=png&auto=webp&v=enabled&s=fd8fe26ff7112c3135e7774636a27de732b5893c

https://preview.redd.it/g1lhwce9bmqa1.png?width=595&format=png&auto=webp&v=enabled&s=ad7550b09ffc7c2f2eb18fe0178d5b4a43ddc4e7

​

71

Comments

You must log in or register to comment.

Peantoo t1_je5iks3 wrote

I understand semantic segmentation but it's been a while since I've tinkered in the computer vision space. Can you explain the push for panoptic segmentation and why you think it's a valuable technology? What can it accomplish that really good semantic segmentation can't?

3

Technical-Vast1314 OP t1_je5oqm5 wrote

OK, panoptic segmentation means doing two kinds of segmentation task together: semantic segmentation and instance segmentation. The semantic segmentation can only segment things like "sky", "car", "person", but it's hard to segment each instance. And instance segmentation is like object detection, which means it will predict a box with a mask on an instance~

6

GFrings t1_je8i7ro wrote

Isn't semantic segmentation made redundant by the instance segmentation? Or is there a difference in coverage for the two tasks, in terms of the ground truth labels?

0