People Jason Corso (PI), Caiming Xiong, David Johnson
Past Members: Albert Chen

Funding: DARPA Computer Science Study Group (CSSG) (HR0011-09-1-0022 and N10AP20032).

This project is kicking off in July 2010.

We propose a revolutionary new approach to data analysis and modeling for computer vision called Active Clustering. Whereas traditional methods in machine learning typically require input from the user before commencing computation and have no subsequent interaction, our approach seeks dynamic input from the user during processing. In comparison to traditional supervised approaches which require extensive up-front effort from the user, in our case, the user will not be required to label large amounts of data. Rather, during processing we will ask simple questions of the user that let us adapt our underlying representation of the sample space. Furthermore, in many defense settings, large amounts of data for a particular target of interest (e.g., the "black Mercedes that is pictured here") may not be available anyway.

In traditional unsupervised, or clustering, methods, the input from the user is in the form of basic assumptions about the sample space. There are two relevant problems with these methods. First, the assumptions typically require some degree of technical know-how on the part of the user. However, many DoD/IC end-user analysts would lack the necessary training to effectively map mission sets to clustering assumptions. Second, without the correct feature space, there is a disparity between the underlying distance function driving the clustering and the user's semantics in most realistic settings. In other words, the samples the clustering algorithm says are similar are in no way tied to the semantics of the user. Our proposed Active Clustering methodology overcomes both of these issues: simple intuitive questions about grouping are asked of the user thereby incorporating his or her semantics and requiring no technical knowledge of how the system works. These high-level questions are tied to the underlying mathematics rigorously.

More recent methods that incorporate the user dynamically, such as Active Learning methods, seek a classifier over a predefined set of classes, which provides convenient mechanisms for selecting which samples to be labeled next by the user. The same convenience does not exist for the clustering (i.e., generative) case because the estimate of uncertainty or information gain is not as readily computed.

Exploitation and forensics comprise the core defense relevance of our proposal with broad applications such as persistent surveillance and urban C2. The VIMEXF Problem is our focus: given a large corpus of video and image data, we want to allow the analyst (level 1, 2 or 3) to quickly search through the video and image data. Possible queries are to search for standard mission elements, or to select the set of clips containing a particular person or feature. Furthermore, the approach must scale well and adapt to new data on-line without full reindexing. We stress the emphasis is on perceptual and semantic content rather than existing meta content such as geospatial coordinates of the field of view.

[1]	C. Xiong, D. M. Johnson, and J. J. Corso. Active clustering with model-based uncertainty reduction. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(1):5--17, 2017. Original Version: ArXiv 1402.1783. [ bib | .pdf ]
[2]	D. M. Johnson, C. Xiong, and J. J. Corso. Semi-supervised nonlinear distance metric learning via forests of max-margin cluster hierarchies. IEEE Transactions on Knowledge and Data Engineering, 28(4):1035--1046, 2016. [ bib | DOI | .pdf ]
[3]	C. Xiong, W. Chen, G. Chen, D. Johnson, and J. J. Corso. Adaptive quantization: An information-based approach to learning binary codes. In Proceedings of SIAM International Conference on Data Mining, 2014. [ bib | code | .pdf ]
[4]	C. Xiong, S. McCloskey, and J. J. Corso. Latent domains for visual domain adaptation. In Proceedings of AAAI Conference on Artificial Intelligence, 2014. [ bib ]
[5]	D. M. Johnson, C. Xiong, J. Gao, and J. J. Corso. Comprehensive cross-hierarchy cluster agreement evaluation. In Proceedings of AAAI Conference on Artificial Intelligence (Late-Breaking Papers Track), 2013. [ bib | code | .pdf ]
[6]	C. Xiong, D. M. Johnson, and J. J. Corso. Uncertainty reduction for active image clustering via a hybrid global-local uncertainty model. In Proceedings of AAAI Conference on Artificial Intelligence (Late-Breaking Papers Track), 2013. [ bib | .pdf ]
[7]	C. Xiong, D. Johnson, R. Xu, and J. J. Corso. Random forests for metric learning with implicit pairwise position dependence. In Proceedings of ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2012. [ bib | slides | code | .pdf ]
[8]	C. Xiong, D. Johnson, and J. J. Corso. Spectral active clustering via purification of the k-nearest neighbor graph. In Proceedings of European Conference on Data Mining, 2012. [ bib | .pdf ]
[9]	C. Xiong, D. Johnson, and J. J. Corso. Efficient max-margin metric learning. In Proceedings of European Conference on Data Mining, 2012. Winner of Best Paper Award at ECDM 2012. [ bib | .pdf ]
[10]	C. Xiong and J. J. Corso. Coaction discovery: Segmentation of common actions across multiple videos. In Proceedings of Multimedia Data Mining Workshop in Conjunction with the ACM SIGKDD Conference on Knowledge Discovery and Data Mining (MDMKDD), 2012. [ bib | .pdf ]