Laura Balzano May 13, 2020

Online Tensor Completion and Tracking

Kyle Gilman and I have a preprint out describing a new algorithm for online tensor completion and tracking. We derive and demonstrate an algorithm that operates on streaming tensor data, such as hyperspectral video collected over time, or chemo-sensing experiments in space and time. Kyle presented his work at the first fully virtual ICASSP, which you can view here. Anyone can register for free to this year’s virtual ICASSP and watch the videos, post questions, and join the discussion. Kyle’s code is also available here.

Filed Under: Code, News, SPADA

Emergent low-rank dynamics at Neurips 2026

October 16, 2026 By Laura Balzano

Our paper on Emergent low-rank training dynamics in nonlinear MLPs has been accepted to Neurips 2026. In the paper, we build from our ICML 2024 paper where we proved that deep linear networks have exactly low-rank training dynamics for the whole trajectory of training for problems with a low-rank target, for example a classification problem with few classes compared to the data dimension. In this current paper, we extend those theoretical results to the setting where the MLP has a nonlinear but smooth activation function. While our theory studies two-layer MLPs where we train only the inner weights, we see empirically that this phenomenon is maintained regardless of the depth of the network. We can then train on a compressed model to get similar accuracy results with a tiny fraction of the trainable parameters. Our key insight in both works is that there is a clever initialization that allows one to train compressed deep networks and maintain similar dynamics (or exactly the same dynamics in the linear case) throughout training. Congratulations especially to Alec Xu for all the hard work he put in to this problem. We hope you’ll come see his poster in Sydney!

Congratulations Can!

August 18, 2026 By Laura Balzano

Congratulations Dr. Can Yaras for successfully defending his PhD in July. His dissertation was titled “Low-dimensional Structures of Learning and Computation in Deep Learning,” including work that is already making an impact in the crowded field of deep learning efficiency and algorithm understanding. He will be moving on to a postdoc at Princeton Language and Intelligence.

Congratulations Alex!

July 10, 2026 By Laura Balzano

I’m so proud of Dr. Alex Ritchie for his excellent PhD defense! His dissertation is titled “Covariate Structure as Weak Supervision in Estimation Problems,” and includes interesting theoretical and empirical results for Supervised PCA as well as learning nonparametric mixtures. We look forward to all the good Dr. Ritchie will certainly do in the world!

Online Tensor Completion and Tracking

Recent News

Emergent low-rank dynamics at Neurips 2026

Congratulations Can!

Congratulations Alex!

© Copyright