Information Theory and
Machine Learning Lab

We are a research group headed by Prof. Jy-yong Sohn in the Department of Applied Statistics at Yonsei University.

We focus on research topics in the intersection of information theory and machine learning. More broadly, we delve into various topics in machine learning and artificial intelligence, using mathematical tools from information theory, optimization, learning theory, and probability & statistics. Current research topics include

• Foundation Models (including Large Language Models)

• Representation Learning

• Efficient Machine Learning

For the details of each research topic, please check this page. If you are interested in joining our group, please contact me. 

Selected Preprints/Publications (the full list is in here)

• [TMLR'24] Mini-Batch Optimization of Contrastive Loss

• [UAI'24] Memorization Capacity for Additive Fine-Tuning

• [AISTATS'24] Analysis of Using Sigmoid Loss for Contrastive Learning

• [ICML'23] Looped Transformers as Programmable Computers

• [ICLR'23] Equal Improvability: A New Fairness Notion Considering the Long-term Impact

• [AAAI'23] Can We Find Strong Lottery Tickets in Generative Models?

Selected Awards & Grants

• NRF Korea, Basic Research Lab (기초연구실), 2024 - 2027

• NRF Korea, Outstanding Young Scientist (우수신진), 2024 - 2027

• Excellence in Teaching Award, Yonsei University, 2023

Selected Invited Talks

• Theory of Contrastive Learning [KIAS, 2024 Sep]

• Theory of Foundation Models [KSIAM, 2024 Aug]

• Theory of Efficient Machine Learning [KIAS, 2023 Dec]

• Large Language Models (LLMs) [Yonsei Institute of Data Science, 2023 April] [Linq, 2023 July] [KICS, 2024 Feb]

• Generative Models [ETRI, 2023 June] [KAIA, 2023 July]