Junyeol Ryu

MS student @ SNU CSE

Summary

Hello :) I am Junyeol Ryu [Pronunciation: "jun-yall" ("yeol" as in Good morning "yall" ☀️)].

I am a Masters student in CSE at the Seoul National University. I am also a prospective PhD student for fall 2025.

I research in parallel computing; building high performance systems that efficiently orchestrate accelerators, storages, and networks.

I am currently working on following topics:
  • Training large language models with limited heterogenous hardware
  • Optimizing GPU collective communication library for deep learning

I am fortunate to be advised by advised by Professor Jaejin Lee and be part of Thunder Research Group.

Publications

SPipe: A Fast and Compact Offloading for Training Large Language Models

Preprint, 2024

TCCL: Discovering Better Communication Paths for PCIe GPU Clusters

ASPLOS, 2024

Heehoon Kim, Junyeol Ryu, and Jaejin Lee

Network Contention-Aware Cluster Scheduling with Reinforcement Learning

ICPADS, 2023

Junyeol Ryu and Jeongyoon Eo

Domestic publications

A Fast and Scalable Generative Model Inference on Distributed Multi-GPU Environment

Korean Computing Congress, 2023

Junyeol Ryu, Jinpyo Kim, and Jaejin Lee

Investigating Contention Sensitivity of DL Training Workloads in Shared GPU Cluster

Korean Software Congress, 2022

Junyeol Ryu and Byung-Gon Chun

Experience

Graduate Student Research Assistant

Thunder Research Group, SNU
Mar 2023 - Present

Advisor: Prof. Jaejin Lee

Building fast and compact software systems for deep learning. Currently working on SPipe, an LLM training system that offloads to external storage but achieves even higher performance! Participated in TCCL, a GPU collective communication library on PCIe-only GPU cluster that achieves up to 2.07x improved efficiency. Open-sourced FastGen, the step-by-step optimization of CUDA GEMM kernel that achieves 80.9% performance of closed-source cuBLAS.

Keywords:

  • MLSys
  • LLM
  • Training
  • HPC
  • Open Source

Graduate Student Research Assistant

Software Platform Lab, SNU
Sep 2022 - Feb 2023
Mar 2022 - Aug 2022 (Research Intern)

Advisor: Prof. Byung-Gon Chun

Focused on efficient scheduling of deep learning jobs in GPU clusters. Created two GPU cluster managers, GPack and DeepShare, which propose resouce-efficient packing using lightweight DNN and network contention mitigation using RL, respectively.

Keywords:

  • MLSys
  • Training
  • Cluster Management
  • Scheduling
  • Open Source

Software Engineer Intern

FriendliAI
Jan 2022 - Feb 2022

Participated in prototype web client development of Periflow, a serving engine for LLMs.

Keywords:

  • LLM
  • Inference

Software Engineer

Waffle Studio
Mar 2021 - Mar 2022

Created Guam, a team-matching mobile app that pairs programmers, project managers, and designers. Led frontend team until successful deployment on app markets and left.

Keywords:

  • Software Engineering
  • Mobile

Software Engineer

Vanilla Bridge
Jul 2020 - Dec 2020

Participated in Vanilla bridge, a dating app with emphasis on credibility by human matchmaker-based system. Focused on data-driven DevOps for service optimization by introducing data analysis with collected in-app user experience data.

Keywords:

  • Software Engineering
  • Mobile
  • Startup

Education

  • MS, Computer Science and Engineering
    (In progress)
    Seoul National University
    Sep 2022 - Present
  • BS, Computer Science and Engineering
    Summa cum laude
    Seoul National University
    Mar 2016 - Aug 2022

Proficiency

Languages

  • C++, Python
  • C, CUDA, OpenMP, OpenCL, MPI
  • Dart, JavaScript, TypeScript, Ruby

Tools and Frameworks

  • PyTorch
  • Django, Flutter, React, Vue
  • AWS, Firebase, BigQuery

Others

  • Commandline
  • GitHub
  • Open Source

Honors & Awards

Teaching

Community Service

  • Auxiliary police
    Served mandatory military service in Republic of Korea
    Mar 2018 - Dec 2019

English Proficiency

Interests

  • Computer Systems
  • Deep Learning
  • Chess