SUMMARY.md

Table of contents

• Introduction

Paper List

• Systems for ML
  • Data Processing
  • Deep Learning Training
  • Resource Scheduler
  • Model Serving
  • Large Language Model (LLM)
  • Diffusion Models
  • Deep Learning Recommendation Model (DLRM)
  • Mixture of Experts (MoE)
  • Hyper-Parameter Tuning (HPO)
  • Reinforcement Learning (RL)
  • Deep Learning Compiler
  • Deep Learning Framework
  • Cloud-Edge Collaboration
• ML for Systems
• Artificial Intelligence (AI)
  • Diffusion Models
  • Language Models
  • Deep Learning Recommendation Model (DLRM)
• Hardware Virtualization
  • GPU Sharing
• Resource Disaggregation
  • GPU Disaggregation
  • Memory Disaggregation
• Resource Fragmentation
• Cloud Computing
  • Sky Computing
  • Serverless Computing
  • Spot Instances
• Remote Direct Memory Access (RDMA)
• Research Skills
• Miscellaneous

Reading Notes

• Conference
  • NSDI 2025
  • ASPLOS 2025
  • EuroSys 2025
  • NeurIPS 2024
  • SoCC 2024
  • HotNets 2024
  • SC 2024
  • SOSP 2024
  • VLDB 2024
  • SIGCOMM 2024
  • ICML 2024
  • ATC 2024
  • OSDI 2024
  • ISCA 2024
  • CVPR 2024
  • MLSys 2024
  • ASPLOS 2024
    • SpotServe: Serving generative large language models on preemptible instances
  • EuroSys 2024
    • Orion: Interference-aware, fine-grained GPU sharing for ML applications
  • NSDI 2024
  • NeurIPS 2023
  • SC 2023
    • Interference-aware multiplexing for deep learning in GPU clusters: A middleware approach
  • SoCC 2023
  • SOSP 2023
    • UGache: A unified GPU cache for embedding-based deep learning
  • SIGCOMM 2023
  • HotChips 2023
  • ICML 2023
  • ATC 2023
    • Accelerating Distributed MoE Training and Inference with Lina
    • SmartMoE: Efficiently Training Sparsely-Activated Models ...
    • Beware of Fragmentation: Scheduling GPU-Sharing Workloads with Fragmentation Gradient Descent
  • OSDI 2023
  • HotOS 2023
  • SIGMOD 2023
  • ISCA 2023
  • MLSys 2023
  • EuroSys 2023
  • NSDI 2023
    • Shepherd: Serving DNNs in the wild
    • Understanding RDMA microarchitecture resources for performance isolation
    • Skyplane: Optimizing transfer cost and throughput using cloud-aware overlays
    • Shockwave: Fair and efficient cluster scheduling for dynamic adaptation in machine learning
  • ASPLOS 2023
    • TPP: Transparent page placement for CXL-enabled tiered-memory
    • EVStore: Storage and caching capabilities for scaling embedding tables in deep recommendation system
    • Lucid: A non-intrusive, scalable and interpretable scheduler for deep learning training jobs
  • SC 2022
  • SoCC 2022
    • ESCHER: Expressive scheduling with ephemeral resources
    • Serving unseen deep learning model with near-optimal configurations: A fast adaptive search approach
  • SIGCOMM 2022
    • Multi-resource interleaving for deep learning training
  • ATC 2022
    • PilotFish: Harvesting Free Cycles of Cloud Gaming with Deep Learning Training
    • Memory Harvesting in Multi-GPU Systems with Hierarchical Unified Virtual Memory
    • Whale: Efficient Giant Model Training over Heterogeneous GPUs
    • DVABatch: Diversity-aware Multi-Entry Multi-Exit Batching for Efficient Processing of DNN Service...
    • Serving Heterogeneous Machine Learning Models on Multi-GPU Servers with Spatio-Temporal Sharing
    • SOTER: Guarding Black-box Inference for General Neural Networks at the Edge
    • Direct access, high-performance memory disaggregation with DirectCXL
  • OSDI 2022
    • Orca: A distributed serving system for transformer-based generative models
    • Microsecond-scale preemption for concurrent GPU-accelerated DNN inferences
    • Looking beyond GPUs for DNN scheduling on multi-tenant clusters
  • IPDPS 2022
    • DGSF: Disaggregated GPUs for serverless functions
  • EuroSys 2022
    • Slashing the disaggregation tax in heterogeneous data centers with FractOS
  • NSDI 2022
  • SoCC 2021
  • ATC 2021
    • Zico: Efficient GPU memory sharing for concurrent DNN training
  • OSDI 2021
    • Pollux: Co-adaptive cluster scheduling for goodput-optimized deep learning
  • SOSP 2021
    • HeMem: Scalable Tiered Memory Management for Big Data Applications and Real NVM
  • EuroSys 2021
    • Take it to the limit: Peak prediction-driven resource overcommitment in datacenters
  • HotOS 2021
    • From cloud computing to sky computing
  • NSDI 2021
  • OSDI 2020
    • A unified architecture for accelerating distributed DNN training in heterogeneous GPU/CPU clusters
    • HiveD: Sharing a GPU cluster for deep learning with guarantees
  • ATC 2020
    • Serverless in the wild: Characterizing and optimizing the serverless workload
  • EuroSys 2020
  • ASPLOS 2020
  • MLSys 2020
  • SoCC 2020
    • Elastic Parameter Server Load Distribution in Deep Learning Clusters
  • HPDC 2020
    • KubeShare: A framework to manage GPUs as first-class and shared resources in container cloud
  • CLUSTER 2019
  • EuroSys 2019
  • NSDI 2019
  • IWQoS 2019
    • Who limits the resource efficiency of my datacenter: An analysis of Alibaba datacenter traces
  • SIGCOMM 2018
    • Revisiting network support for RDMA
  • OSDI 2018
    • Ray: A distributed framework for emerging AI applications
  • EuroSys 2018
    • Medea: Scheduling of long running applications in shared production clusters
  • ISPA/IUCC/BDCloud/SocialCom/SustainCom 2018
    • GaiaGPU: Sharing GPUs in container clouds
  • SoCC 2017
    • SLAQ: Quality-driven scheduling for distributed machine learning
  • ASPLOS 2017
    • Neurosurgeon: Collaborative intelligence between the cloud and mobile edge
  • NSDI 2017
    • Clipper: A low-latency online prediction serving system
  • CLUSTER 2014
    • Evaluating job packing in warehouse-scale computing
• Journal
  • IEEE Transactions on Cloud Computing
    • 2021
      • Gemini: Enabling multi-tenant GPU sharing based on kernel burst estimation
  • ACM Computing Surveys
    • 2017
      • GPU virtualization and scheduling methods: A comprehensive survey
  • ACM SIGCOMM Computer Communication Review (CCR)
    • 2021
      • Data-driven Networking Research: models for academic collaboration with industry
    • 2007
      • How to Read a Paper
  • Communications of the ACM
    • 2015
      • Why Google stores billions of lines of code in a single repository
• Miscellaneous
  • arXiv
    • 2024
      • Efficiently programming large language models using SGLang
    • 2023
      • HexGen: Generative inference of foundation model over heterogeneous decentralized environment
      • High-throughput generative inference of large language models with a single GPU
    • 2022
      • DisaggRec: Architecting disaggregated systems for large-scale personalized recommendation
      • A case for disaggregation of ML data processing
      • Singularity: Planet-scale, preemptive and elastic scheduling of AI workloads
      • Aryl: An elastic cluster scheduler for deep learning
    • 2016
      • Wide & deep learning for recommender systems
      • Training deep nets with sublinear memory cost
  • MSR Technical Report
    • 2011
      • Heuristics for vector bin packing

About Myself

• Academic Profile
• Personal Blog (in Chinese)