Code for my Proceedings of IEEE paper with Yuejie Chi and Yue Lu, Streaming PCA and Subspace Tracking: the Missing Data Case can be found here.
April 2, 2022
Code for my Proceedings of IEEE paper with Yuejie Chi and Yue Lu, Streaming PCA and Subspace Tracking: the Missing Data Case can be found here.
October 16, 2026 By Laura Balzano
Our paper on Emergent low-rank training dynamics in nonlinear MLPs has been accepted to Neurips 2026. In the paper, we build from our ICML 2024 paper where we proved that deep linear networks have exactly low-rank training dynamics for the whole trajectory of training for problems with a low-rank target, for example a classification problem with few classes compared to the data dimension. In this current paper, we extend those theoretical results to the setting where the MLP has a nonlinear but smooth activation function. While our theory studies two-layer MLPs where we train only the inner weights, we see empirically that this phenomenon is maintained regardless of the depth of the network. We can then train on a compressed model to get similar accuracy results with a tiny fraction of the trainable parameters. Our key insight in both works is that there is a clever initialization that allows one to train compressed deep networks and maintain similar dynamics (or exactly the same dynamics in the linear case) throughout training. Congratulations especially to Alec Xu for all the hard work he put in to this problem. We hope you’ll come see his poster in Sydney!
August 18, 2026 By Laura Balzano
Congratulations Dr. Can Yaras for successfully defending his PhD in July. His dissertation was titled “Low-dimensional Structures of Learning and Computation in Deep Learning,” including work that is already making an impact in the crowded field of deep learning efficiency and algorithm understanding. He will be moving on to a postdoc at Princeton Language and Intelligence.
July 10, 2026 By Laura Balzano
I’m so proud of Dr. Alex Ritchie for his excellent PhD defense! His dissertation is titled “Covariate Structure as Weak Supervision in Estimation Problems,” and includes interesting theoretical and empirical results for Supervised PCA as well as learning nonparametric mixtures. We look forward to all the good Dr. Ritchie will certainly do in the world!
Copyright © 2026 · Streamline Child Theme on Genesis Framework · WordPress · Log in