-
Where to Hitch? Finding Small, Rare Parts Across Long Spatiotemporal Contexts
-
Cheng-Lin Hsieh and Zilin Wang and Stella X. Yu and Enrique Corona and Sandeep Sasidharan and Erol Dogan Sumer},
-
Neural Information Processing Systems Workshop on Long Context Foundation Models, Atlanta, Georgia, 12 December 2026
-
Paper
-
Abstract
-
Foundation models struggle to find a small, rare, and visually plain part in a long, high-resolution video. Their fixed input resolution makes processing every frame in full detail costly, while downsizing hides the part. Naming the part does not single it out either: the concept is rare, and its appearance alone is not distinctive. We study this problem for hitch assist, which locates a trailer's coupler as the vehicle backs toward it. The coupler covers a median 0.2\% of the image, and text-prompted models often box a larger structure. We introduce \textbf{HitchIt}, a training-free method that selects the spatial and temporal context a frozen video foundation model sees. HitchIt specifies the part with boxed examples from other videos and searches a few late views, where the coupler is large. In each view, it narrows the search from the trailer through its tongue to the coupler: the trailer and tongue are easier to find, and each crop enlarges the coupler in the model's input and excludes distractors. It then tracks the best box backward and forward through the video. On 20 densely annotated videos, HitchIt reaches 0.627 mean box overlap (mIoU), versus at most $0.067$ for text-prompted models, including SAM3 on the same late views. On 67 separate videos, part-by-part search raises mIoU from $0.513$ to $0.645$ over full-image search. We will release our dense coupler annotations.
-
Keywords
-
small object localization, part localization, spatiotemporal context, vision foundation models, video object tracking
|