Stella X. Yu : Papers / Google Scholar

Pose-Aware Self-Supervised Learning with Viewpoint Trajectory Regularization Jiayun Wang and Yubei Chen and Stella X. Yu European Conference on Computer Vision, Milan, Italy, 28 September-4 October 2024 Paper | Slides | Code | arXiv Abstract Learning visual features from unlabeled images has proven successful for semantic categorization, often by mapping different views of the same object to the same feature to achieve recognition invariance. However, visual recognition involves not only identifying what an object is but also understanding how it is presented. For example, seeing a car from the side versus head-on is crucial for deciding whether to stay put or jump out of the way. While unsupervised feature learning for downstream viewpoint reasoning is important, it remains under-explored, partly due to the lack of a standardized evaluation method and benchmarks. We introduce a new dataset of adjacent image triplets obtained from a viewpoint trajectory, without any semantic or pose labels. We bench- mark both semantic classification and pose estimation accuracies on the same visual feature. Additionally, we propose a viewpoint trajectory reg- ularization loss for learning features from unlabeled image triplets. Our experiments demonstrate that this approach helps develop a visual rep- resentation that encodes object identity and organizes objects by their poses, retaining semantic classification accuracy while achieving emer- gent global pose awareness and better generalization to novel objects. Our dataset and code are available at http://pwang.pw/trajSSL/. Keywords self-supervised learning, pose estimation, trajectory