Advait Gadhikar

I am a third year PhD student at the CISPA Helmholtz Center for Information Security in Saarbrücken, advised by Dr. Rebekka Burkholz. I work on building provable algorithms for deep learning and am currently interested in algorithms related to sparsity in neural nets. Specifically I am interested in the Lottery Ticket Hypothesis and how it can help identify the underlying structure of a learnt network.

I have completed my Masters from the ECE Department, Carnegie Mellon University in Pittsburgh, PA. During this time I worked on building Optimization algorithms with theoretical guarantees for training Machine Learning models in large scalable systems with the Optimization, Probability and Learning Group (OPAL) advised by Prof. Gauri Joshi.

Prior to my Masters at CMU, I was an undergraduate at BITS Pilani in India. As an undergraduate, I was fortunate to work with Prof Rajiv Soundararajan at the Indian Institute of Science on my Bachelor Thesis. Here, I worked on designing Image Quality Assessment models that are able to perceive images like the Human Visual System.

Email  /  LinkedIn  /  GitHub  /  CV

Feel free to get in touch with me, always looking forward to insightful discussions!

• [March 2025] Our work 'Attention Is All You Need For Mixture-of-Depths Routing' is accepted at the SCOPE workshop in ICLR 2025.
• [June 2024] I will be spending the next six months with Bosch AI, developing Sparse Mixture of Expert models for efficient inference.
• [Jan 2024] Our paper on 'Masks, Signs, and Learning Rate Rewinding' is accepted at ICLR 2024 for a spotlight presentation.
• [April 2023] Our paper on 'Why Random Pruning Is All You Need to Start Sparse' is accepted at ICML 2023.
• [October 2022] Check out our work on Dynamical Isometry for Residual Networks .
• [April 2022] Started my PhD with Rebekka Burkholz
• [September 2021] Our work with Prof. Gauri Joshi's group at CMU, 'Leveraging Spatial and Temporal Correlations in Sparsified Mean Estimation', got accepted at NEURIPS 2021!
• [Jan 2021] Our paper AdaQuantFL on Adaptive Quantization in Federated Learning got accepted in ICASSP 2021.
• [Dec 2020] Graduated with MS in ECE from CMU
• [Oct 2020] Submitted our work on Adaptive Gradient Quantization in Federated Learning to ICASSP 2021 and is under review
• [August 2020] Interned over the Summer with the Video Systems team at Qualcomm on video denoising algorithms
• [May 2020] Joined OPAL as a Graduate Research Assistant
• [Jan 2020] Started as a Teaching Assistant for Prof Bhiksha's course on Introduction to Deep Learning
• [August 2019] Started as a Masters student at CMU
• [May 2019] Graduated from BITS Pilani, Goa Campus
• [Dec 2018] Completed Bachelor Thesis at the Indian Institute of Science