For the complete lists of papers, see all publications page. Shown below is the list of selected papers. You can also check my google scholar.
For the complete lists of papers, see all publications page. Shown below is the list of selected papers. You can also check my google scholar.
Overarching Theme: Motivated by the current success/failure of machine learning (ML) fields, we aim at
building mathematical frameworks to better understand the success of existing ML algorithms
devising algorithms that overcome the limitations of existing methods
Current Focus: Pretrained large models are having surprising zero-shot and few-shot performances on various downstream tasks. This huge success inspired the concept of foundation models, which contains recent large-scale models including Large Language Models (LLMs) and Vision Language Models (VLMs).
We are interested in theory, algorithms, and applications of foundation models. (* means corresponding author)
Theoretical Analysis of Foundation Models
Measuring Representational Shifts in Continual Learning: A Linear Transformation Perspective [arxiv] [openreview]
J. Kim, Y. Kim and J. Sohn*
ICML 2025
On the Similarities of Embeddings in Contrastive Learning [arxiv] [openreview]
C. Lee, S. Lim, K. Lee and J. Sohn*
ICML 2025
A Theoretical Framework for Preventing Class Collapse in Supervised Contrastive Learning [arxiv] [pmlr]
C. Lee, J. Oh, K. Lee and J. Sohn*
AISTATS 2025
Mini-Batch Optimization of Contrastive Loss [arxiv] [github]
K. Sreenivasan, K. Lee, J.-G. Lee, A. Lee, J. Cho, J. Sohn, D. Papailiopoulos and K. Lee
TMLR 2024 (preliminary result presented in ICLR 2023 Workshop)
Memorization Capacity for Additive Fine-Tuning with Small ReLU Networks [arxiv] [pmlr]
J. Sohn, D. Kwon, S. An and K. Lee
UAI 2024
Analysis of Using Sigmoid Loss for Contrastive Learning [arxiv] [pmlr]
C. Lee, J. Chang and J. Sohn*
AISTATS 2024
Algorithms for Foundation Models
Enhancing Compositional Reasoning in CLIP via Reconstruction and Alignment of Text Descriptions
J. Kwon, K. Min and J. Sohn*
NeurIPS 2025
Task-Aware Routing of Representations for Invariant–Equivariant Self-Supervised Learning
J. Jeon, H. Jang, J. Sohn and K. Lee
NeurIPS 2025
Improving Multi-lingual Alignment Through Soft Contrastive Learning [paper]
M. Park, S. Choi, C. Choi, J. Kim* and J. Sohn*
NAACL 2024 Workshop
Re-Ex: Revising after Explanation Reduces the Factual Errors in LLM Responses [arxiv]
J. Kim, J. Lee, Y. Chang, C. Choi, J. Kim and J. Sohn*
ICLR 2024 Workshop
Can Separators Improve Chain-of-Thought Prompting? [arxiv]
Y. Park, H. Kim, C. Choi, J. Kim and J. Sohn*
arXiv 2024
LIFT: Language-Interfaced FineTuning for Non-Language Machine Learning Tasks [arxiv] [twitter] [github]
T. Dinh, Y. Zeng, R. Zhang, Z. Lin, M. Gira, S. Rajput, J. Sohn, D. Papailiopoulos and K. Lee
NeurIPS 2022
Utilizing Language-Image Pretraining for Efficient and Robust Bilingual Word Alignment [arxiv] [twitter] [github]
T. Dinh, J. Sohn, S. Rajput, T. Ossowski, Y. Ming, J. Hu, D. Papailiopoulos and K. Lee
Findings of EMNLP 2022
Applications of Foundation Models on Various Domains (Healthcare, Finance, Law, Entertainment, etc.)
Aligning Large Language Models for Cognitive Behavioral Therapy: A Proof-Of-Concept Study [paper]
Y. Kim, C. Choi, S. Cho, J. Sohn* and B.-H. Kim*
Frontiers in Psychiatry, 2025.
FinDER: Financial Dataset for Question Answering and Evaluating Retrieval-Augmented Generation [arxiv]
C. Choi, J. Kwon, J. Ha, H. Choi, C. Kim, Y. Lee, J. Sohn and A. Lopez-Lira
ICLR 2025 Workshop
Aligning Large Language Models for Enhancing Psychiatric Interviews through Symptom Delineation and Summarization [arxiv] [github] [paper]
J. So, J. Chang, E. Kim, J. Na, J. Choi, J. Sohn*, B.-H. Kim* and S. H. Chu*
JMIR Formative Research, 2024.
ERD: A Framework for Improving LLM Reasoning for Cognitive Distortion Classification [arxiv]
S. Lim, Y. Kim, C.-H. Choi, J. Sohn* and B.-H. Kim*
NAACL 2024 Workshop
Retrieval-based Evaluation for LLMs: A Case Study in Korean Legal QA [paper]
C. Ryu, S. Lee, S. Pang, C. Choi, H. Choi, M. Min and J. Sohn*
EMNLP 2023 Workshop
Efficient ML
Recent successful machine learning models are using tremendous amount of parameters & data, which require huge storage/computation burden. How can we reduce the training & inference cost by devising efficient algorithms? We focus on research topics including data pruning/distillation and model compression
Can We Find Strong Lottery Tickets in Generative Models? [project page]
S. Yeo, Y. Jang, J. Sohn, D. Han and J. Yoo
AAAI 2023
Rare Gems: Finding Lottery Tickets at Initialization [arxiv] [twitter]
K. Sreenivasan, J. Sohn, L. Yang, M. Grinde, A. Nagle, H. Wang, K. Lee and D. Papailiopoulos
NeurIPS 2022
Finding Everything within Random Binary Networks
K. Sreenivasan, S. Rajput, J. Sohn and D. Papailiopoulos
AISTATS 2022
Trustworthy ML
We aim at developing robust/fair/private machine learning algorithms.
Equal Improvability: A New Fairness Notion Considering the Long-term Impact [arxiv] [github]
O. Guldogan, Y. Zeng, J. Sohn, R. Pedarsani, K. Lee
ICLR 2023
GenLabel: Mixup Relabeling using Generative Models [arxiv] [github]
J. Sohn, L. Shang, H. Chen, J. Moon, D. Papailiopoulos and K. Lee
ICML 2022
Breaking Fair Binary Classification with Optimal Flipping Attacks
C. Jo, J. Sohn and K. Lee
ISIT 2022
Communication-Computation Efficient Secure Aggregation for Federated Learning [arxiv]
B. Choi, J. Sohn, D. -J. Han and J. Moon
on arXiv
Distributed ML
We focus on how machine learning algorithms should be evoled to make use of data/computation distributed over the network.
BufferGA:Buffer-based Gradient Adjustment for Continual Federated Learning
S. Dai, B. Chen, J. Sohn, S. Alam, R. Balakrishnan, S. Banerjee, N, Himayat and K. Lee
MLSys 2023 Workshop
Election Coding for Distributed Learning: Protecting SignSGD against Byzantine Attacks
J. Sohn, D. -J. Han, B. Choi, and J. Moon
NeurIPS 2020
Attack of the Tails: Yes, You Really Can Backdoor Federated Learning
H. Wang, K. Sreenivasan, S. Rajput, H. Vishwakarma, S. Agarwal, J. Sohn, K. Lee, and D. Papailiopoulos
NeurIPS 2020
Hierarchical Coding for Distributed Computing
H. Park, K. Lee, J. Sohn, C. Suh and J. Moon
IEEE ISIT 2018
Information/Communication Theory
Capacity of Clustered Distributed Storage
J. Sohn, B. Choi, S. W. Yoon and J. Moon
IEEE Transactions on Information Theory, 2019
(conference version won the Best Paper Award)
Secure clustered distributed storage against eavesdropping
B. Choi, J. Sohn, S. W. Yoon and J. Moon
IEEE Transactions on Information Theory, 2019
On Reusing Pilots Across Interfering Cells in Massive MIMO
J. Sohn, S. W. Yoon and J. Moon
IEEE Transactions on Wireless Communications, 2017
Pilot Reuse Strategy Maximizing the Weighted-Sum-Rate in Massive MIMO Systems
J. Sohn, S. W. Yoon and J. Moon
IEEE Journal on Selected Areas in Communications, 2017