{"id":939,"date":"2024-06-25T14:28:56","date_gmt":"2024-06-25T14:28:56","guid":{"rendered":"https:\/\/web.eecs.umich.edu\/~girasole\/?p=939"},"modified":"2024-06-25T14:28:56","modified_gmt":"2024-06-25T14:28:56","slug":"code-for-deep-lora","status":"publish","type":"post","link":"https:\/\/web.eecs.umich.edu\/~girasole\/?p=939","title":{"rendered":"Code for Deep LoRA"},"content":{"rendered":"\n<p>Our ICML paper &#8220;<a href=\"https:\/\/openreview.net\/forum?id=uDkXoZMzBv\">Compressible Dynamics in Deep Overparameterized Low-Rank Learning &amp; Adaptation<\/a>&#8221; shows that it is possible to get the benefits of deep overparameterization without increasing the number of trainable parameters. The code for our experiments can be found at Can&#8217;s github site: <a href=\"https:\/\/github.com\/cjyaras\/deep-lora-transformers\">https:\/\/github.com\/cjyaras\/deep-lora-transformers<\/a>. We welcome any thoughts and questions if you use and adapt our code for your problem!<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Our ICML paper &#8220;Compressible Dynamics in Deep Overparameterized Low-Rank Learning &amp; Adaptation&#8221; shows that it is possible to get the benefits of deep overparameterization without increasing the number of trainable parameters. The code for our experiments can be found at Can&#8217;s github site: https:\/\/github.com\/cjyaras\/deep-lora-transformers. We welcome any thoughts and questions if you use and adapt [&hellip;]<\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[3],"tags":[],"_links":{"self":[{"href":"https:\/\/web.eecs.umich.edu\/~girasole\/index.php?rest_route=\/wp\/v2\/posts\/939"}],"collection":[{"href":"https:\/\/web.eecs.umich.edu\/~girasole\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/web.eecs.umich.edu\/~girasole\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/web.eecs.umich.edu\/~girasole\/index.php?rest_route=\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/web.eecs.umich.edu\/~girasole\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=939"}],"version-history":[{"count":1,"href":"https:\/\/web.eecs.umich.edu\/~girasole\/index.php?rest_route=\/wp\/v2\/posts\/939\/revisions"}],"predecessor-version":[{"id":940,"href":"https:\/\/web.eecs.umich.edu\/~girasole\/index.php?rest_route=\/wp\/v2\/posts\/939\/revisions\/940"}],"wp:attachment":[{"href":"https:\/\/web.eecs.umich.edu\/~girasole\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=939"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/web.eecs.umich.edu\/~girasole\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=939"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/web.eecs.umich.edu\/~girasole\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=939"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}