Frontier LLM Architectures MOC

This is the map for a college-course-level learning sphere on frontier LLM architectures. The goal is not only to know model names, but to learn how frontier labs think: mechanisms, scaling constraints, empirical taste, systems bottlenecks, evaluation, and research judgment.

How To Use This Sphere

1. Read Course Roadmap Frontier LLM Research first.
2. Read Architecture Concept Graph Frontier LLMs so the whole stack connects in your head.
3. Build the core model from Transformer Block Anatomy, Transformer Math and Implementation Deep Dive, and Attention Mechanics and KV Cache.
4. Learn why frontier work is dominated by scale through Scaling Laws and Compute Optimal Training and Training Optimization and Stability Deep Dive.
5. Study the modern pressure points: Mixture of Experts Architectures, Long Context and Efficient Sequence Models, and Frontier Model Systems and Inference.
6. Learn post-training through Post Training Alignment and Reasoning.
7. Learn how claims are tested through Evaluation Benchmarks and Scientific Method.
8. Use Paper Reading Ladder Frontier LLMs as the canonical reading sequence.
9. Keep Glossary Frontier LLM Architectures open while reading papers.
10. Use Open Research Questions Frontier LLM Architectures and Implementation Roadmap to Frontier Lab Readiness to turn learning into research taste.

Core Notes

• Course Roadmap Frontier LLM Research
• Math and ML Foundations for Frontier LLMs
• Architecture Concept Graph Frontier LLMs
• Researcher Skill Stack for Frontier Labs
• Transformer Block Anatomy
• Transformer Math and Implementation Deep Dive
• Attention Mechanics and KV Cache
• Scaling Laws and Compute Optimal Training
• Training Optimization and Stability Deep Dive
• Data Tokenization and Pretraining Objective
• Mixture of Experts Architectures
• Long Context and Efficient Sequence Models
• Post Training Alignment and Reasoning
• Frontier Model Systems and Inference
• Evaluation Benchmarks and Scientific Method
• Frontier Model Case Studies
• Implementation Roadmap to Frontier Lab Readiness
• Paper Reading Ladder Frontier LLMs
• Research Memo Template for LLM Papers
• Glossary Frontier LLM Architectures
• Open Research Questions Frontier LLM Architectures

Adjacent Learning Spheres

• Frontier Lab Readiness MOC
• ML Systems for Frontier Models MOC
• Mechanistic Interpretability MOC
• Evaluation and Benchmarking MOC
• Agents and Tool Use MOC
• Multimodal Foundation Models MOC
• AI Safety and Security MOC
• Research Engineering Practices

The Big Picture

Frontier LLM architecture is mostly the study of constraints:

• How do we turn compute into capability?
• How do we make attention, memory, data, and optimization scale?
• How do we train models that can learn broad world models without collapsing into memorization, instability, or unhelpful behavior?
• How do we serve those models cheaply enough that people can use them?
• How do we create reliable reasoning behavior from next-token training plus post-training?

The answer is not one trick. It is a stack:

• Data Tokenization and Pretraining Objective defines what the model sees and what loss it optimizes.
• Transformer Block Anatomy defines the basic compute graph.
• Attention Mechanics and KV Cache defines how information flows across tokens.
• Scaling Laws and Compute Optimal Training defines the economic envelope.
• Mixture of Experts Architectures defines one route to sparse parameter scale.
• Long Context and Efficient Sequence Models defines the memory/context frontier.
• Post Training Alignment and Reasoning shapes behavior after pretraining.
• Frontier Model Systems and Inference determines whether any of it is affordable in practice.
• Evaluation Benchmarks and Scientific Method determines whether we actually know the change helped.
• Architecture Concept Graph Frontier LLMs shows how these layers feed back into each other.

The Connected Stack

Use this sequence when studying any new paper:

1. Objective/data: what distribution and loss created the model?
2. Architecture: what computation changed?
3. Optimization: can this train stably?
4. Scale: what happens at fixed compute, fixed data, fixed latency, or fixed dollars?
5. Systems: does the hardware make the idea practical?
6. Post-training: is the observed behavior from the base model or elicitation?
7. Evaluation: is the measurement clean, contaminated, or proxy-only?

That sequence is the mental checklist behind Research Memo Template for LLM Papers.

Research North Star

To become lab-ready, aim for this ability:

Given a new architecture paper, identify the bottleneck it attacks, reconstruct the mechanism, estimate what scaling regime it helps, name its failure modes, and design one clean experiment that would falsify the claim.

That is the difference between reading about LLMs and thinking like a researcher.