Frontier Lab Learning Notes

Self-study notes for learning frontier LLM architectures and the neighboring skills needed to become research-capable: ML systems, mechanistic interpretability, evaluation, agents/tool use, multimodal models, AI safety/security, and research engineering.

These notes are written as an Obsidian-style knowledge graph, but they also work as normal GitHub Markdown.

Start Here

1. Frontier Lab Readiness MOC
2. Frontier LLM Architectures MOC
3. Course Roadmap Frontier LLM Research
4. Implementation Roadmap to Frontier Lab Readiness
5. Paper Reading Ladder Frontier LLMs

Learning Spheres

• Frontier LLM Architectures
• ML Systems for Frontier Models
• Mechanistic Interpretability
• Evaluation and Benchmarking
• Agents and Tool Use
• Multimodal Foundation Models
• AI Safety and Security
• Research Engineering Practices

Suggested Study Path

Phase 1: Foundations

• Math and ML Foundations for Frontier LLMs
• Transformer Block Anatomy
• Transformer Math and Implementation Deep Dive
• Attention Mechanics and KV Cache

Phase 2: Scaling and Training

• Scaling Laws and Compute Optimal Training
• Training Optimization and Stability Deep Dive
• Data Tokenization and Pretraining Objective

Phase 3: Frontier Pressure Points

• Mixture of Experts Architectures
• Long Context and Efficient Sequence Models
• Frontier Model Systems and Inference
• Post Training Alignment and Reasoning

Phase 4: Research Taste

• Evaluation Benchmarks and Scientific Method
• Open Research Questions Frontier LLM Architectures
• Research Memo Template for LLM Papers
• Frontier Model Case Studies

Primary Source Anchors

The notes link to primary sources such as:

• Attention Is All You Need
• Scaling Laws for Neural Language Models
• Training Compute-Optimal Large Language Models
• FlashAttention
• Switch Transformers
• InstructGPT
• Direct Preference Optimization
• vLLM / PagedAttention
• Transformer Circuits
• ReAct
• Toolformer
• CLIP

Notes On Format

• Obsidian wikilinks were converted to standard relative Markdown links so they work in GitHub and the GitHub mobile app.
• GitHub relative links are included throughout the notes for easy mobile reading.
• The repository intentionally contains only the frontier-learning notes, not the rest of the private vault.