For the complete lists of papers, see all publications page. Shown below is the list of selected papers.

Current Research Topics

Overarching Theme: Motivated by the current success/failure of machine learning (ML) fields, we aim at 

• building mathematical frameworks to better understand the success of existing ML algorithms

• devising algorithms that overcome the limitations of existing methods

Research topics are categorized as below. (* means corresponding author)

1. Theoretical Analysis on Foundation Models
   Pretrained large models are having surprising zero-shot and few-shot performances on various downstream tasks. This huge success inspired the concept of foundation models, which contains recent large-scale models including GPT, CLIP and BERT. We focus on understanding the recent success of foundation models. 

   • Mini-Batch Optimization of Contrastive Loss [arxiv] [github]
     K. Sreenivasan, K. Lee, J.-G. Lee, A. Lee, J. Cho, J. Sohn, D. Papailiopoulos and K. Lee
     TMLR 2024 (preliminary result presented in ICLR 2023 Workshop)

   • Memorization Capacity for Additive Fine-Tuning

J. Sohn, D. Kwon, S. An and K. Lee

UAI 2024

• Analysis of Using Sigmoid Loss for Contrastive Learning [arxiv]
  C. Lee, J. Chang and J. Sohn*
  AISTATS 2024

1. • Looped Transformers as Programmable Computers  [arxiv] [twitter]
     A. Giannou, S. Rajput, J. Sohn, K. Lee, J. D. Lee and D. Papailiopoulos
     ICML 2023

2. Empirical Findings on Foundation Models
   Foundation models including Large Language Models (LLMs) have a significant impact on various fields. We focus on empirical attemps on understanding the possibility and limitations of foundation models, and  overcoming the limitations of them.

   • Improving Multi-lingual Alignment Through Soft Contrastive Learning
     M. Park, S. Choi, C. Choi, J. Kim* and J. Sohn*
     NAACL 2024 Workshop

   • ERD: A Framework for Improving LLM Reasoning for Cognitive Distortion Classification [arxiv]
     S. Lim, Y. Kim, C.-H. Choi, J. Sohn* and B.-H. Kim*
     NAACL 2024 Workshop

   • Re-Ex: Revising after Explanation Reduces the Factual Errors in LLM Responses [arxiv]
     J. Kim, J. Lee, Y. Chang, C. Choi, J. Kim and J. Sohn*
     ICLR 2024 Workshop

   • Can Separators Improve Chain-of-Thought Prompting? [arxiv]
     Y. Park, H. Kim, C. Choi, J. Kim and J. Sohn*
     arXiv 2024

   • Retrieval-based Evaluation for LLMs: A Case Study in Korean Legal QA [paper]
     C. Ryu, S. Lee, S. Pang, C. Choi, H. Choi, M. Min and J. Sohn*
     EMNLP 2023 Workshop

   • LIFT: Language-Interfaced FineTuning for Non-Language Machine Learning Tasks [arxiv] [twitter] [github]
     T. Dinh, Y. Zeng, R. Zhang, Z. Lin, M. Gira, S. Rajput, J. Sohn, D. Papailiopoulos and K. Lee
     NeurIPS 2022 

   • Utilizing Language-Image Pretraining for Efficient and Robust Bilingual Word Alignment [arxiv] [twitter] [github]
     T. Dinh, J. Sohn, S. Rajput, T. Ossowski, Y. Ming, J. Hu, D. Papailiopoulos and K. Lee
     Findings of EMNLP 2022

3. Efficient ML
   Recent successful machine learning models are using tremendous amount of parameters & data, which require huge storage/computation burden. How can we reduce the training & inference cost by devising efficient algorithms? We focus on research topics including data pruning/distillation and model compression

   • Can We Find Strong Lottery Tickets in Generative Models? [project page]
     S. Yeo, Y. Jang, J. Sohn, D. Han and J. Yoo
     AAAI 2023

   • Rare Gems: Finding Lottery Tickets at Initialization [arxiv] [twitter]
     K. Sreenivasan, J. Sohn, L. Yang, M. Grinde, A. Nagle, H. Wang, K. Lee and D. Papailiopoulos
     NeurIPS 2022

   • Finding Everything within Random Binary Networks
     K. Sreenivasan, S. Rajput, J. Sohn and D. Papailiopoulos
     AISTATS 2022

Past Research Topics

1. Trustworthy ML
   We aim at developing robust/fair/private machine learning algorithms.

   • Equal Improvability: A New Fairness Notion Considering the Long-term Impact [arxiv] [github]
     O. Guldogan, Y. Zeng, J. Sohn, R. Pedarsani, K. Lee
     ICLR 2023

   • GenLabel: Mixup Relabeling using Generative Models [arxiv] [github]
     J. Sohn, L. Shang, H. Chen, J. Moon, D. Papailiopoulos and K. Lee
     ICML 2022

   • Breaking Fair Binary Classification with Optimal Flipping Attacks
     C. Jo, J. Sohn and K. Lee
     ISIT 2022

   • Communication-Computation Efficient Secure Aggregation for Federated Learning [arxiv]
     B. Choi, J. Sohn, D. -J. Han and J. Moon
     on arXiv

2. Distributed ML
   We focus on how machine learning algorithms should be evoled to make use of data/computation distributed over the network.

   • BufferGA:Buffer-based Gradient Adjustment for Continual Federated Learning
     S. Dai, B. Chen, J. Sohn, S. Alam, R. Balakrishnan, S. Banerjee, N, Himayat and K. Lee
     MLSys 2023 Workshop

   • Election Coding for Distributed Learning: Protecting SignSGD against Byzantine Attacks
     J. Sohn, D. -J. Han, B. Choi, and J. Moon
     NeurIPS 2020

   • Attack of the Tails: Yes, You Really Can Backdoor Federated Learning
     H. Wang, K. Sreenivasan, S. Rajput, H. Vishwakarma, S. Agarwal, J. Sohn, K. Lee, and D. Papailiopoulos
     NeurIPS 2020

   • Hierarchical Coding for Distributed Computing
     H. Park, K. Lee, J. Sohn, C. Suh and J. Moon
     IEEE ISIT 2018

3. Information/Communication Theory

   • Capacity of Clustered Distributed Storage
     J. Sohn, B. Choi, S. W. Yoon and J. Moon
     IEEE Transactions on Information Theory, 2019
     (conference version won the Best Paper Award)

   • Secure clustered distributed storage against eavesdropping
     B. Choi, J. Sohn, S. W. Yoon and J. Moon
     IEEE Transactions on Information Theory, 2019

   • On Reusing Pilots Across Interfering Cells in Massive MIMO
     J. Sohn, S. W. Yoon and J. Moon
     IEEE Transactions on Wireless Communications, 2017

   • Pilot Reuse Strategy Maximizing the Weighted-Sum-Rate in Massive MIMO Systems
     J. Sohn, S. W. Yoon and J. Moon
     IEEE Journal on Selected Areas in Communications, 2017