For the complete lists of papers, see all publications page. Shown below is the list of selected papers.
Current Research Topics
Overarching Theme: Motivated by the current success/failure of ML fields, we aim at
building mathematical frameworks to better understand the success of existing ML algorithms
devising algorithms that overcome the limitations of existing methods
Research topics are categorized as below. (* means corresponding author)
Efficient ML
Recent successful machine learning models are using tremendous amount of parameters & data, which require huge storage/computation burden. How can we reduce the training & inference cost by devising efficient algorithms? We focus on research topics including data pruning/distillation and model compression (structured/unstructured pruning, quantization, low-rank approximation, model distillation).Can We Find Strong Lottery Tickets in Generative Models? [project page]
S. Yeo, Y. Jang, J. Sohn, D. Han and J. Yoo
AAAI 2023Rare Gems: Finding Lottery Tickets at Initialization [arxiv] [twitter]
K. Sreenivasan, J. Sohn, L. Yang, M. Grinde, A. Nagle, H. Wang, K. Lee and D. Papailiopoulos
NeurIPS 2022Finding Everything within Random Binary Networks
K. Sreenivasan, S. Rajput, J. Sohn and D. Papailiopoulos
AISTATS 2022
Transformers & Foundation Models (e.g., GPT, CLIP)
Pretrained large models are having surprising zero-shot and few-shot performances on various downstream tasks. This huge success inspired the concept of foundation models, which contains recent large-scale models including GPT, CLIP and BERT. We focus on understanding & exploiting foundation models. Another related topic we are interested in is transformer, an architecture that enabled the huge success of neural networks in NLP.ERD: A Framework for Improving LLM Reasoning for Cognitive Distortion Classification [arxiv]
S. Lim, Y. Kim, C.-H. Choi, J. Sohn* and B.-H. Kim*
NAACL 2024 WorkshopRe-Ex: Revising after Explanation Reduces the Factual Errors in LLM Responses [arxiv]
J. Kim, J. Lee, Y. Chang, C. Choi, J. Kim and J. Sohn*
ICLR 2024 WorkshopCan Separators Improve Chain-of-Thought Prompting? [arxiv]
Y. Park, H. Kim, C. Choi, J. Kim and J. Sohn*
arXiv 2024Retrieval-based Evaluation for LLMs: A Case Study in Korean Legal QA [paper]
C. Ryu, S. Lee, S. Pang, C. Choi, H. Choi, M. Min and J. Sohn*
EMNLP 2023 WorkshopLooped Transformers as Programmable Computers [arxiv] [twitter]
A. Giannou, S. Rajput, J. Sohn, K. Lee, J. D. Lee and D. Papailiopoulos
ICML 2023LIFT: Language-Interfaced FineTuning for Non-Language Machine Learning Tasks [arxiv] [twitter] [github]
T. Dinh, Y. Zeng, R. Zhang, Z. Lin, M. Gira, S. Rajput, J. Sohn, D. Papailiopoulos and K. Lee
NeurIPS 2022Utilizing Language-Image Pretraining for Efficient and Robust Bilingual Word Alignment [arxiv] [twitter] [github]
T. Dinh, J. Sohn, S. Rajput, T. Ossowski, Y. Ming, J. Hu, D. Papailiopoulos and K. Lee
Findings of EMNLP 2022
Representation Learning
Learning a good representation is a key for the success of artificial intelligence. We focus on the theory and practice of the representations learned by self-supervised learning (e.g., contrastive learning and masked language modeling).
Past Research Topics
Trustworthy ML
We aim at developing robust/fair/private machine learning algorithms.Equal Improvability: A New Fairness Notion Considering the Long-term Impact [arxiv] [github]
O. Guldogan, Y. Zeng, J. Sohn, R. Pedarsani, K. Lee
ICLR 2023GenLabel: Mixup Relabeling using Generative Models [arxiv] [github]
J. Sohn, L. Shang, H. Chen, J. Moon, D. Papailiopoulos and K. Lee
ICML 2022Breaking Fair Binary Classification with Optimal Flipping Attacks
C. Jo, J. Sohn and K. Lee
ISIT 2022Communication-Computation Efficient Secure Aggregation for Federated Learning [arxiv]
B. Choi, J. Sohn, D. -J. Han and J. Moon
on arXiv
Distributed ML
We focus on how machine learning algorithms should be evoled to make use of data/computation distributed over the network.BufferGA:Buffer-based Gradient Adjustment for Continual Federated Learning
S. Dai, B. Chen, J. Sohn, S. Alam, R. Balakrishnan, S. Banerjee, N, Himayat and K. Lee
MLSys 2023 WorkshopElection Coding for Distributed Learning: Protecting SignSGD against Byzantine Attacks
J. Sohn, D. -J. Han, B. Choi, and J. Moon
NeurIPS 2020Attack of the Tails: Yes, You Really Can Backdoor Federated Learning
H. Wang, K. Sreenivasan, S. Rajput, H. Vishwakarma, S. Agarwal, J. Sohn, K. Lee, and D. Papailiopoulos
NeurIPS 2020Hierarchical Coding for Distributed Computing
H. Park, K. Lee, J. Sohn, C. Suh and J. Moon
IEEE ISIT 2018
Information Theory for Distributed Storage
Capacity of Clustered Distributed Storage
J. Sohn, B. Choi, S. W. Yoon and J. Moon
IEEE Transactions on Information Theory, 2019
(conference version won the Best Paper Award)Secure clustered distributed storage against eavesdropping
B. Choi, J. Sohn, S. W. Yoon and J. Moon
IEEE Transactions on Information Theory, 2019
Signal Processing for Communication Systems
On Reusing Pilots Across Interfering Cells in Massive MIMO
J. Sohn, S. W. Yoon and J. Moon
IEEE Transactions on Wireless Communications, 2017Pilot Reuse Strategy Maximizing the Weighted-Sum-Rate in Massive MIMO Systems
J. Sohn, S. W. Yoon and J. Moon
IEEE Journal on Selected Areas in Communications, 2017