ML Systems for Frontier Models MOC
ML systems is the study of making model training and inference actually run: fast, stable, memory-efficient, distributed, and affordable.
If you want frontier-lab credibility, this is one of the highest-ROI areas.
Why It Matters
Frontier models are limited by:
- GPU memory.
- Memory bandwidth.
- Interconnect bandwidth.
- Kernel efficiency.
- Distributed communication.
- Checkpointing.
- KV-cache size.
- Inference latency.
- Cost per useful token.
Architecture ideas often fail because they are systems-hostile.
Core Topics
GPU Basics
Learn:
- FLOPs versus memory bandwidth.
- GPU memory hierarchy.
- Tensor cores.
- Kernel launch overhead.
- Batch size and occupancy.
- Mixed precision.
Connects to:
Training Systems
Learn:
- Data parallelism.
- Tensor parallelism.
- Pipeline parallelism.
- Sequence/context parallelism.
- Expert parallelism.
- Activation checkpointing.
- Optimizer-state partitioning.
Primary papers:
Inference Systems
Learn:
- Prefill versus decode.
- KV cache.
- Continuous batching.
- Paged attention.
- Quantization.
- Speculative decoding.
- Time to first token.
- Tokens per second.
Primary paper:
Kernel Programming
Learn:
- PyTorch performance basics.
torch.compile.- Triton.
- Profiling.
- Custom attention kernels.
Primary source:
Starter Project
Build a transformer inference profiler:
- Load or implement a small decoder-only transformer.
- Measure prefill latency and decode latency separately.
- Compare no-cache versus KV-cache generation.
- Estimate KV memory by layer/head/context length.
- Try MHA versus GQA if available.
- Plot latency and memory versus sequence length.
Deliverable:
- A short memo: "Where does inference time and memory go?"
Papers / Resources
What Good Looks Like
You can:
- Explain why decode is often memory-bound.
- Estimate KV-cache memory.
- Read a GPU profiler trace.
- Explain why MoE is a communication problem.
- Compare throughput and latency honestly.
- Know when an architecture improvement is erased by serving cost.