I am honored to have received the NSF CAREER award for a proposal on optimization methods and theory for the joint formulation of dimension reduction and clustering. You can read about the award here in the UM press release and also here on the NSF website. Dimension reduction and clustering are arguably the two most critical problems in unsupervised machine learning; they are used universally for data exploration and understanding. Often dimension reduction is used before clustering (or vice versa) to lend tractability to the modeling algorithm. It’s more typical in real data to see clusters each with their own low-dimensional structure, and so a joint formulation is of great interest. I look forward to working toward this end in the next stage of my career.

Stay ahead with Ledger Live’s latest enhancements, ensuring top-notch security for your digital assets.

Army Young Investigator

I am very excited that my project “Mathematics for Learning Nonlinear Generalizations of Subspace Models in High Dimensions” has won the Army Young Investigator award! Subspace models are widely used due to simplicity and ease of analysis. However, while these linear models are very powerful in many high-dimensional data contexts, they also often miss out on important nonlinearities in real data. This project aims to extend recent advances in signal processing to the single-index model and the nonlinear variety model. Read the department’s announcement here.

Postdoc Opportunity at the University of Michigan

to begin in spring 2019.


Please email Laura Balzano <girasole@umich.edu> with the subject “Joining the Balzano lab — postdoc 2019” if you are interested.

We are seeking a postdoc who is interested in applying machine learning techniques to real-time dynamic data analysis. While machine learning has advanced significantly over the last decade, its application to dynamic time-varying data is still in its infancy. This project will focus on three ML areas: online learning, stochastic gradient methods, and streaming PCA. We will work on theory to understand how the standard approaches behave when the data are time-varying, develop appropriate models for time-varying data, and develop novel approaches along with convergence theory. Our main applications focus will be power systems engineering and computer vision. In power systems, we will develop methodologies to infer the real-time behavior of aggregations of distributed energy resources from hierarchical, heterogeneous, and incomplete measurements of power system quantities. In computer vision, we will develop real-time algorithms for object tracking and activity recognition in video.

Optimally Weighted PCA for High-dimensional Heteroscedastic Data

Today I had the opportunity to speak about very recent results by my student David Hong (joint work also with Jeff Fessler) in analyzing asymptotic recovery guarantees for weighted PCA for high-dimensional heteroscedastic data. In the paper we recently posted online, we have asymptotic analysis (as both the number of samples and dimension of the problem grow to infinity, but converge to a fixed constant) of the recovery for weighted PCA components, amplitudes, and scores. Those recovery expressions allow us to find weights that give optimal recovery, and the weights turn out to be a very simple expression involving only the noise variance and the PCA amplitudes. To learn more, watch my talk here, and let us know if you have any questions!

The problem with this method is that after the detailed plan for relocating your body to Dvortsovaya Street has been drawn up, we have nothing more to talk about and all the norms of public ethics demand that we part ways and part ways for good. So the wisest thing to do aff app is to figure out in which direction the beauty is pointing her feet, and try to tail her. Since you are on the road, let her take you to the second right turn towards the Georgian embassy.

AFOSR Young Investigator

I have great news that my AFOSR Young Investigator proposal was accepted for funding. My proposal was focused on time-varying low-rank factorization models, and various ways of solving a variety of related non-convex problem formulations.  Read more about it here.   I look forward to the contributions we will be able to make with the support of AFOSR.


Ensemble K-Subspaces

Yesterday I gave a talk on Subspace Clustering using Ensemble methods at the Simons Institute. See the video here!

This is work with John Lipor, David Hong, and Yan Shuo Tan. Our related paper has been just updated on the arxiv. Our key observation was that, while K-Subspaces (KSS) works poorly overall and depends heavily on initialization, it still seems to give partially good clustering information. We therefore use it as a “weak clusterer” and combine ensembles of KSS (EKSS) by averaging the co-association/affinity matrices. This works extremely well, both in simulation and on real data, and also in theory. We were able to show that EKSS gives correct clustering in a variety of common cases: e.g. for subspaces with bounded affinity, and with noisy data and missing data. Our theory generalizes theory of the Thresholded Subspace Clustering algorithm to show that any algorithm that produces an affinity matrix that is an approximation to a monotonic function of absolute inner products will give correct clustering. This general theory should be broadly applicable to many geometric approaches to subspace clsutering.

Improving K-Subspaces via Coherence Pursuit

John Lipor, Andrew Gitlin, Bioashuai Tao, and I have a new paper, “Improving K-Subspaces via Coherence Pursuit,” to be published in the Journal of Special Topics in Signal Processing issue “Robust Subspace Learning and Tracking: Theory, Algorithms, and Applications.” In it we present a new subspace clustering algorithm, Coherence Pursuit – K-Subspaces (CoP-KSS). Here is the code for CoP-KSS and for our figures. Our paper considers specifically the PCA step in K-Subspaces, where a best-fit subspace estimate is determined from a (possibly incorrect) clustering. When a given cluster has points from multiple low-rank subspaces, PCA is not a robust approach. We replace that step with Coherence Pursuit, a new algorithm for Robust PCA. We prove that Coherence Pursuit indeed can recover the “majority” subspace when data from other low-rank subspaces is contaminating the cluster. In this paper we also prove — to the best of our knowledge, for the first time — that the K-Subspaces problem is NP-hard, and indeed even NP-hard to approximate within any finite factor for large enough subspace rank.

In turn, roulette, which is now one of the most popular casino table games all over the world, appeared in the 18th century in France, in the gambling houses of Paris. Translated https://pin-ups-casino.com from French, the word “roulette” means “small wheel”, it became incredibly popular throughout Europe and after some time came to the United States, where it became one of the favorite entertainment of Americans.

The origins of poker are also not completely clear, because, like many other games of chance, poker, most likely, also developed over several centuries, taking shape from different card games. Some argue that poker-like gambling was invented in the 17th century in Persia, while others say that the famous game of today was inspired by the French game Poque. The popularity of this game grew rather slowly until the 70s. of the last century, no world poker tournaments were held in Las Vegas. But the greatest recognition of this game was provided by the opportunity to gamble on the Internet when online poker appeared.


New paper in Journal of Multivariate Analysis

Congratulations to my student David Hong (and his co-advisor Jeff Fessler) for our published article in the Journal of Multivariate Analysis, titled “Asymptotic performance of PCA for high-dimensional heteroscedastic data.” Heteroscedastic data, where different data points are of differing quality (precisely, have different noise variance), are common in so many interesting big data problems. Sensor network data, medical imaging using historical data, and astronomical imaging are just a few examples. PCA is known to be the maximum likelihood estimate for data with additive Gaussian noise of a single variance across all the data points. This work investigates the performance of PCA when that homoscedastic noise assumption is violated. We give precise predictions for the recovery of subspaces and singular values in a spiked/planted model, and show that vanilla PCA (perhaps unsurprisingly) has suboptimal subspace recovery when the data are heteroscedastic. 

There is also no clear answer to the question of the origin of one of the most popular modern casino table games – blackjack – most often France is considered to be its country of origin. It is believed that the 17th century French game vingt-et-un is the direct ancestor of blackjack, and it appeared in the United States along with the first colonists from France. The name blackjack is of American origin, and it was associated with special promotions that were held at casinos in Nevada in the 30s. XX century. In order to attract new customers, higher https://pinup-wiki.com/ odds were paid if the player won with a combination that included a jack of black suit (clubs or spades) and an ace of spades. In English, the jack of black suit (clubs or spades) is called “black Jack of Clubs” or “black Jack of Spades”.

Publications Update

We have had many exciting publications in the last several months.

My student Dejiao Zhang and I worked with Mario Figueiredo and two other Michigan students on applying OWL regularization in deep networks. The intuition is that since OWL can tie correlated regressors, it should be able to do the same in deep nets that experience a high degree of co-adaptation (and correlation) of nodes in the network. Dejiao presented our paper Learning to Share: Simultaneous Parameter Tying and Sparsification for Deep Learning at ICLR last month and we will present Simultaneous Sparsity and Parameter Tying for Deep Learning using Ordered Weighted L1 Regularization at SSP next month.

With my colleague Johanna Mathieu and her student Greg Ledva, we published a paper in Transactions on Power Systems studying Real-Time Energy Disaggregation of a Distribution Feeder’s Demand Using Online Learning. The work leverages recent results in dynamic online learning where classes of dynamical models are used to apply online learning to the time-varying signal setting. This work can leverage existing sensing structure to improve prediction of distributed energy resources, demand-responsive electric loads and residential solar generation. We also have a book chapter in Energy Markets and Responsive Grids that was written also with my student Zhe Du.

Greg Ongie, David Hong, Dejiao Zhang, and I have been working on adaptive sampling for subspace estimation. If one has a matrix in memory that is large and difficult to access, but you want to compute a low-rank approximation of that matrix, one way is to sketch it by reading only parts of the matrix and computing an approximation. Our paper Enhanced Online Subspace Estimation Via Adaptive Sensing describes an adaptive sampling scheme to do exactly that, and using that scheme along with the GROUSE subspace estimation algorithm, we gave global convergence guarantees to the true underlying low-rank matrix. We will also present Online Estimation of Coherent Subspaces with Adaptive Sampling at SSP next month, which constrains the adaptive samples to be entry-wise and sees similar improvements.

Rounding it out, Zhe Du will be presenting our work with Necmiye Ozay on A Robust Algorithm for Online Switched System Identification at the SYS ID conference in July, and Bob Malinas and David Hong will present our work with Jeff Fessler on Learning Dictionary-Based Unions of Subspaces for Image Denoising at EUSIPCO in September. This spring Amanda Bower presented our work with Lalit Jain on The Landscape of Nonconvex Quadratic Feasibility, studying the minimizers for a non-convex formulation of the preference learning problem; and next week Naveen Murthy presents our work with Greg Ongie and Jeff Fessler on Memory-efficient Splitting Algorithms for Large-Scale Sparsity Regularized Optimization at the CT Meeting. Last fall Greg Ongie, Saket Dewangan, Jeff Fessler and I had a paper Online Dynamic MRI Reconstruction via Robust Subspace Tracking at GlobalSIP, pursuing the interesting idea of online subspace tracking for time-varying signals.

So many exciting research directions that we will continue to pursue!

Congratulations John!

Congratulations to Dr. John Lipor for successfully defending his PhD thesis in September! The title of his work is “Sensing Structured Signals with Active and Ensemble Methods.” In January he will start as Assistant Professor in the Portland State University ECE Department.

Malian refugees and, having moved to the United States, became the first participant in the Miss Minnesota contest, wearing a hijab and burkini. In the early 1990s, restrictions became less strict: Kate Moss appeared with her fragile figure and the appearance of a ragged boy. By the standards of the catwalk https://beautypositive.org/, she was considered low – 170 centimeters. An ordinary British teenager , Kate did not have the particular grace and stature that gave many other models a grandeur. Her stellar ad campaign for Calvin Klein heralded the end of an era of long-legged gazelle dominance on the runways.