Mingchen Li

Senior Research Scientist at Meta

Research interests

My research spans machine learning and optimization, with a focus on foundation models, AI safety, network architecture, robust neural network training, semi-supervised learning and bilevel optimization. Recently I have been working on LLM-based model oversight, sub-quadratic language modeling, and the design of hybrid LLM architectures, especially convolution-based architectures.

Experience

Senior Research Scientist

2024 – present Meta, Ranking AI
  • Ads ranking and recommendation modeling.

Software Engineer Intern & Student Researcher

Jun – Nov 2022 Google, RankLab team
  • Noise detection on imbalanced datasets.

Software Engineer Intern

Jun – Sep 2021 Google, YouTube Shorts ranking team
  • Improved the user-profiling model to enhance the YouTube Shorts recommendation system.

Education

Ph.D., Electrical and Computer Engineering

2023 – 2024 University of Michigan, Ann Arbor · Advisor: Samet Oymak

Dissertation: Toward Effective Neural Architectures and Algorithms for Generalizable Deep Learning

Ph.D. Candidate, Computer Science

2018 – 2023 University of California, Riverside · Advisor: Samet Oymak · GPA 3.93/4.0

Transferred to the University of Michigan with my advisor in 2023.

B.S., Information Security

2013 – 2017 Fudan University, School of Computer Science and Technology · Advisor: Yanqiu Chen

Selected research

On the Power of Convolution-Augmented Transformer

2025 AAAI 2025
  • Proposed the Convolution-augmented Attention Transformer (CAT) to improve retrieval and summarization in LLMs.
  • Proved that a single CAT layer solves associative recall and copying tasks while enjoying length generalization.
  • Demonstrated CAT's benefits in real language modeling, for both model training and length extrapolation.

Provable Noise Robustness of Early-Stopped Neural Networks

2020 AISTATS 2020540+ citations
  • Showed that large neural networks can overfit to label noise, which hurts accuracy.
  • Proved that a large network becomes provably robust to label noise when trained with early stopping.

FedNest: Federated Bilevel, Minimax, and Compositional Optimization

2022 ICML 2022Oral · top 2% acceptance110+ citations
  • Proposed FedNest, a federated gradient-based method for nested optimization problems.
  • Proved convergence rates under data heterogeneity, with variants for bilevel, minimax and compositional optimization.
  • Experiments on hyperparameter and hyper-representation learning show its benefit in practice.

AutoBalance: Optimized Loss Functions for Imbalanced Data

2021 NeurIPS 2021110+ citations
  • Built a bilevel optimization framework that automatically designs the training loss, blending accuracy and fairness objectives for long-tailed and group-imbalanced data.
  • The learned per-class/per-group treatment outperforms state-of-the-art approaches in extensive evaluations.

Generalization Guarantees for Neural Architecture Search with Train-Validation Split

2021 ICML 202130+ citations
  • Showed that the validation loss of a near-minimal validation set is indicative of the true test loss, for continuous search spaces used by differentiable NAS.
  • Established generalization bounds for NAS (activation search) and proved that train-validation selection finds the best architecture even when the model overfits the training data.

Citation counts from Google Scholar, October 2026; see Google Scholar for current numbers.

Publications

Conference and journal papers

  1. On the Power of Convolution-Augmented Transformer. M. Li, X. Zhang, Y. Huang, S. Oymak. AAAI 2025. arXiv
  2. Selective Attention: Enhancing Transformer through Principled Context Control. X. Zhang, X. Chang, M. Li, A. Roy-Chowdhury, J. Chen, S. Oymak. NeurIPS 2024. 45 citations arXiv
  3. Class-Attribute Priors: Adapting Optimization to Heterogeneity and Fairness Objective. X. Zhang, M. Li, J. Chen, C. Thrampoulidis, S. Oymak. AAAI 2024. arXiv
  4. FedNest: Federated Bilevel, Minimax, and Compositional Optimization. D. A. Tarzanagh, M. Li, C. Thrampoulidis, S. Oymak. ICML 2022. Oral · top 2% 110+ citations arXiv
  5. AutoBalance: Optimized Loss Functions for Imbalanced Data. M. Li, X. Zhang, C. Thrampoulidis, J. Chen, S. Oymak. NeurIPS 2021. 110+ citations arXiv
  6. Generalization Guarantees for Neural Architecture Search with Train-Validation Split. S. Oymak, M. Li, M. Soltanolkotabi. ICML 2021. 30+ citations arXiv
  7. On the Marginal Benefit of Active Learning: Does Self-Supervision Eat Its Cake? Y. C. Chan, M. Li, S. Oymak. ICASSP 2021. 35+ citations arXiv
  8. Gradient Descent with Early Stopping is Provably Robust to Label Noise for Overparameterized Neural Networks. M. Li, M. Soltanolkotabi, S. Oymak. AISTATS 2020. 540+ citations arXiv
  9. Generalization, Adaptation and Low-Rank Representation in Neural Networks. S. Oymak, Z. Fabian, M. Li, M. Soltanolkotabi. Asilomar Conference on Signals, Systems, and Computers 2019.
  10. Robust 3-D Human Detection in Complex Environments with a Depth Camera. L. Tian, M. Li, Y. Hao, J. Liu, G. Zhang, Y. Q. Chen. IEEE Transactions on Multimedia (TMM), 20(9), 2018. 45+ citations
  11. Robust Human Detection with Super-Pixel Segmentation and Random Ferns Classification Using RGB-D Camera. L. Tian, M. Li, G. Zhang, J. Zhao, Y. Q. Chen. IEEE ICME 2017.
  12. Reliably Detecting Humans in Crowded and Dynamic Environments Using RGB-D Camera. L. Tian, G. Zhang, M. Li, J. Liu, Y. Q. Chen. IEEE ICME 2016.

Peer-reviewed workshops and technical reports

  1. Algorithmic Oversight for Deceptive Reasoning. E. O. Taga, M. Li, Y. Chen, S. Oymak. Red Teaming GenAI Workshop: What Can We Learn from Adversaries?, 2024.
  2. Augmenting Federated Learning with Pretrained Transformers (FedYolo). X. Zhang, M. Li, X. Chang, J. Chen, A. Roy-Chowdhury, A. Suresh, S. Oymak. NeurIPS 2023 Workshop on Federated Learning in the Age of Foundation Models. arXiv
  3. Federated Multi-Sequence Stochastic Approximation with Local Hypergradient Estimation. D. A. Tarzanagh, M. Li, P. Sharma, S. Oymak. arXiv 2023. arXiv
  4. Provable and Efficient Continual Representation Learning. Y. Li, M. Li, M. S. Asif, S. Oymak. arXiv 2022. arXiv
  5. Exploring Weight Importance and Hessian Bias in Model Pruning. M. Li, Y. Sattar, C. Thrampoulidis, S. Oymak. arXiv 2020. arXiv
  6. Generalization Guarantees for Neural Networks via Harnessing the Low-Rank Structure of the Jacobian. S. Oymak, Z. Fabian, M. Li, M. Soltanolkotabi. ICML 2019 Workshop on Understanding and Improving Generalization in Deep Learning. 130+ citations arXiv

Skills and academic service

  • Area Chair: ICML 2026 NeurIPS 2026 ICLR 2026
  • Reviewer: NeurIPS, ICML, ICLR, AISTATS, ISIT, KDD
  • Teaching assistant at UC Riverside: Deep Learning, Artificial Intelligence, Data Science
  • ML frameworks: PyTorch, JAX, TensorFlow
  • GPU programming: CUDA, Triton

Awards

Dissertation Year Program (DYP) Fellowship, CS Department, UC Riverside

2022

Second-Class Scholarship, Fudan University twice · top 4 of 32

2015 & 2017