Colloquia - Shu Kong, Open-World Visual Perception, Virtual, 4:25 - 5:25 pm

Monday, December 6, 2021 - 4:25pm to 5:25pm
Event Type: 

Shu Kong - man smiling in front of pebble background

Open-World Visual Perception


Shu Kong is a Postdoctoral Fellow in the Robotics Institute at Carnegie Mellon University, supervised by Prof. Deva Ramanan. He earned a Ph.D. in Computer Science at the University of California at Irvine, advised by Prof. Charless Fowlkes. His research interests span computer vision and machine learning, and their applications including autonomous vehicles and interdisciplinary research. His current research focus is on open-world visual perception. His recent paper on this topic received Best Paper / Marr Prize Honorable Mention at the International Conference on Computer Vision (ICCV) 2021. His latest interdisciplinary research outcome includes a high-throughput pollen analysis system, which was featured by the National Science Foundation as one that "opens a new era of fossil pollen research...greatly enhances the use of pollen data in ecological and evolutionary research".


Visual perception is indispensable for numerous applications. Today's visual perception algorithms are often developed under a closed-world paradigm, which assumes the data distribution and categorical labels are fixed a priori. This is unrealistic in the dynamic, vast, and unpredictable open world. Failing to perceive the open world can cause catastrophic issues. For example, autonomous vehicles should identify an unknown overturned truck or an indiscernible person on a dark night to avoid vehicle collisions or pedestrian casualty; commercial visual recognition software should attend to rarely-occurring observations to avoid making offensive misclassifications for people from underrepresented groups. These scenarios motivate my research topic of open-world visual perception. In this presentation, I will discuss three topics, (1) segmenting every instance, (2) recognizing the unknowns, and (3) detecting what you can’t see. The topics will cover broad aspects including long-tailed distribution, probabilistic and adversarial learning, multimodality, and interdisciplinary research. I will conclude the talk with my future research plan.