Stella X. Yu : Papers / Google Scholar

Unsupervised Hierarchical Semantic Segmentation with Multiview Cosegmentation and Clustering Transformers
Tsung-Wei Ke and Jyh-Jing Hwang and Yunhui Guo and Xudong Wang and Stella X. Yu
IEEE Conference on Computer Vision and Pattern Recognition, New Orleans, Louisiana, 19-24 June 2022
Paper | Slides | Code | arXiv


Unsupervised semantic segmentation aims to discover groupings within and across images that capture object- and view-invariance of a category without external supervision. Grouping naturally has levels of granularity, creating ambiguity in unsupervised segmentation. Existing methods avoid this ambiguity and treat it as a factor outside modeling, whereas we embrace it and desire hierarchical grouping consistency for unsupervised segmentation.

We approach unsupervised segmentation as a pixel-wise feature learning problem. Our idea is that a good representation shall reveal not just a particular level of grouping, but any level of grouping in a consistent and predictable manner. We enforce spatial consistency of grouping and bootstrap feature learning with co-segmentation among multiple views of the same image, and enforce semantic consistency across the grouping hierarchy with clustering transformers between coarse- and fine-grained features.

We deliver the first data-driven unsupervised hierarchical semantic segmentation method called Hierarchical Seg- ment Grouping (HSG). Capturing visual similarity and statistical co-occurrences, HSG also outperforms existing unsupervised segmentation methods by a large margin on five major object- and scene-centric benchmarks.

unsupervised hierarchical semantic segmentation, co-segmentation, clustering transformer,