For the complete lists of papers, see all publications page. Shown below is the list of selected papers.
For the complete lists of papers, see all publications page. Shown below is the list of selected papers.
Overarching Theme: Motivated by the current success/failure of machine learning (ML) fields, we aim at
building mathematical frameworks to better understand the success of existing ML algorithms
devising algorithms that overcome the limitations of existing methods
Research topics are categorized as below. (* means corresponding author)
Theoretical Analysis on Foundation Models
Pretrained large models are having surprising zero-shot and few-shot performances on various downstream tasks. This huge success inspired the concept of foundation models, which contains recent large-scale models including GPT, CLIP and BERT. We focus on understanding the recent success of foundation models.
A Theoretical Framework for Preventing Class Collapse in Supervised Contrastive Learning [arxiv]
C. Lee, J. Oh, K. Lee and J. Sohn*
AISTATS 2025
Mini-Batch Optimization of Contrastive Loss [arxiv] [github]
K. Sreenivasan, K. Lee, J.-G. Lee, A. Lee, J. Cho, J. Sohn, D. Papailiopoulos and K. Lee
TMLR 2024 (preliminary result presented in ICLR 2023 Workshop)
Memorization Capacity for Additive Fine-Tuning with Small ReLU Networks [arxiv]
J. Sohn, D. Kwon, S. An and K. Lee
UAI 2024
Analysis of Using Sigmoid Loss for Contrastive Learning [arxiv]
C. Lee, J. Chang and J. Sohn*
AISTATS 2024
Empirical Findings on Foundation Models
Foundation models including Large Language Models (LLMs) have a significant impact on various fields. We focus on empirical attemps on understanding the possibility and limitations of foundation models, and overcoming the limitations of them.
Improving Multi-lingual Alignment Through Soft Contrastive Learning [paper]
M. Park, S. Choi, C. Choi, J. Kim* and J. Sohn*
NAACL 2024 Workshop
ERD: A Framework for Improving LLM Reasoning for Cognitive Distortion Classification [arxiv]
S. Lim, Y. Kim, C.-H. Choi, J. Sohn* and B.-H. Kim*
NAACL 2024 Workshop
Re-Ex: Revising after Explanation Reduces the Factual Errors in LLM Responses [arxiv]
J. Kim, J. Lee, Y. Chang, C. Choi, J. Kim and J. Sohn*
ICLR 2024 Workshop
Can Separators Improve Chain-of-Thought Prompting? [arxiv]
Y. Park, H. Kim, C. Choi, J. Kim and J. Sohn*
arXiv 2024
Retrieval-based Evaluation for LLMs: A Case Study in Korean Legal QA [paper]
C. Ryu, S. Lee, S. Pang, C. Choi, H. Choi, M. Min and J. Sohn*
EMNLP 2023 Workshop
LIFT: Language-Interfaced FineTuning for Non-Language Machine Learning Tasks [arxiv] [twitter] [github]
T. Dinh, Y. Zeng, R. Zhang, Z. Lin, M. Gira, S. Rajput, J. Sohn, D. Papailiopoulos and K. Lee
NeurIPS 2022
Utilizing Language-Image Pretraining for Efficient and Robust Bilingual Word Alignment [arxiv] [twitter] [github]
T. Dinh, J. Sohn, S. Rajput, T. Ossowski, Y. Ming, J. Hu, D. Papailiopoulos and K. Lee
Findings of EMNLP 2022
Efficient ML
Recent successful machine learning models are using tremendous amount of parameters & data, which require huge storage/computation burden. How can we reduce the training & inference cost by devising efficient algorithms? We focus on research topics including data pruning/distillation and model compression
Can We Find Strong Lottery Tickets in Generative Models? [project page]
S. Yeo, Y. Jang, J. Sohn, D. Han and J. Yoo
AAAI 2023
Rare Gems: Finding Lottery Tickets at Initialization [arxiv] [twitter]
K. Sreenivasan, J. Sohn, L. Yang, M. Grinde, A. Nagle, H. Wang, K. Lee and D. Papailiopoulos
NeurIPS 2022
Finding Everything within Random Binary Networks
K. Sreenivasan, S. Rajput, J. Sohn and D. Papailiopoulos
AISTATS 2022
Trustworthy ML
We aim at developing robust/fair/private machine learning algorithms.
Equal Improvability: A New Fairness Notion Considering the Long-term Impact [arxiv] [github]
O. Guldogan, Y. Zeng, J. Sohn, R. Pedarsani, K. Lee
ICLR 2023
GenLabel: Mixup Relabeling using Generative Models [arxiv] [github]
J. Sohn, L. Shang, H. Chen, J. Moon, D. Papailiopoulos and K. Lee
ICML 2022
Breaking Fair Binary Classification with Optimal Flipping Attacks
C. Jo, J. Sohn and K. Lee
ISIT 2022
Communication-Computation Efficient Secure Aggregation for Federated Learning [arxiv]
B. Choi, J. Sohn, D. -J. Han and J. Moon
on arXiv
Distributed ML
We focus on how machine learning algorithms should be evoled to make use of data/computation distributed over the network.
BufferGA:Buffer-based Gradient Adjustment for Continual Federated Learning
S. Dai, B. Chen, J. Sohn, S. Alam, R. Balakrishnan, S. Banerjee, N, Himayat and K. Lee
MLSys 2023 Workshop
Election Coding for Distributed Learning: Protecting SignSGD against Byzantine Attacks
J. Sohn, D. -J. Han, B. Choi, and J. Moon
NeurIPS 2020
Attack of the Tails: Yes, You Really Can Backdoor Federated Learning
H. Wang, K. Sreenivasan, S. Rajput, H. Vishwakarma, S. Agarwal, J. Sohn, K. Lee, and D. Papailiopoulos
NeurIPS 2020
Hierarchical Coding for Distributed Computing
H. Park, K. Lee, J. Sohn, C. Suh and J. Moon
IEEE ISIT 2018
Information/Communication Theory
Capacity of Clustered Distributed Storage
J. Sohn, B. Choi, S. W. Yoon and J. Moon
IEEE Transactions on Information Theory, 2019
(conference version won the Best Paper Award)
Secure clustered distributed storage against eavesdropping
B. Choi, J. Sohn, S. W. Yoon and J. Moon
IEEE Transactions on Information Theory, 2019
On Reusing Pilots Across Interfering Cells in Massive MIMO
J. Sohn, S. W. Yoon and J. Moon
IEEE Transactions on Wireless Communications, 2017
Pilot Reuse Strategy Maximizing the Weighted-Sum-Rate in Massive MIMO Systems
J. Sohn, S. W. Yoon and J. Moon
IEEE Journal on Selected Areas in Communications, 2017