Free Energy to LLMs

Generative models and probability, from variational free energy to LLM training and inference (in Chinese)

Introduction

An open textbook on generative models, written from the viewpoint of statistical physics and probability and carried from mathematical derivation all the way to engineering practice. For every model it asks four questions: which distribution it represents, which objective it learns with, how it samples, and where the real bottleneck lies.

The book covers VAE, normalizing flows, GAN, diffusion and energy-based models, then Transformer, multimodal models and reinforcement-learning post-training, followed by training systems, inference systems, agent systems and the architecture innovations of DeepSeek and Kimi, and finally returns to generative models in scientific computing. It has about 150 pages in 11 chapters and 2 appendices, with 34 TikZ figures whose sources are released together with the book.

Courses

  • Oct 2026, Graduate School of China Academy of Engineering Physics: three 50-minute lectures. Slides are in the slides/ folder of the repo.

  • Repo: https://github.com/refraction-ray/free-energy-to-llms

  • PDF: https://github.com/refraction-ray/free-energy-to-llms/releases/latest

The text, figures, slides and their sources are released under CC BY-NC-SA 4.0.

refraction-ray/free-energy-to-llms refraction-ray/free-energy-to-llms