

530 episodes
by Enoch H. Kang · English
Cut through the noise. We curate and break down the most important AI papers so you don’t have to.

Self-adapting language models (SEAL) that improve by generating “self-edits” (natural-language instructions) to create their own training data and update their weights, using meta-learning.
12 Oct 2025 · 17m · 8 chapters
The episode explains why “long chain of thought” (long-CoT) training is so expensive for LLMs (quadratic compute from transformer attention over growing context, plus linear KV-cache memory growth),…
12 Oct 2025 · 14m · 6 chapters
The episode explains “Moloch’s bargain” in AI: when LLMs are optimized to win against competitors for short-term audience approval (sales, votes, engagement), long-term goals like truth and safety…
12 Oct 2025 · 17m · 12 chapters
Explains in-context learning (ICL) variability using a top-down “rational analysis” framework: a hierarchical Bayesian model where transformer behavior is a weighted mix of a generalizing predictor…
11 Oct 2025 · 16m · 9 chapters
Why “thinking models” (e.g., DeepSeek R1, Claude 3.7 Sonnet) outperform base LLMs on hard math—arguing that base models already contain core reasoning skills, and post-training mainly teaches…
11 Oct 2025 · 12m · 6 chapters
How post-training instruction tuning can damage a model’s conditional distributional modeling (CDM), reducing in-context steerability, output diversity/coverage, and distributional alignment; and how…
11 Oct 2025 · 16m · 10 chapters
Explains why LLM prompting works using a Bayesian meta-learning view of in-context learning, and compares hard prompt tuning vs prefix tuning (soft prompts), including when prompting fails and how…
11 Oct 2025 · 14m · 6 chapters
Compares in-context learning (ICL) across MLPs, MLP mixers, and transformers on synthetic regression/classification and relational reasoning tasks, focusing on compute efficiency, context length…
11 Oct 2025 · 16m · 10 chapters
Few-shot learning comparison between pre-training (PT) + head fine-tuning and model-agnostic meta-learning (MAML), arguing PT isn’t universally better; results depend on task/data diversity measured…
11 Oct 2025 · 21m · 11 chapters
Agentic Context Engineering (ACE) for self-improving LLM agents—how to adapt task context over time without catastrophic forgetting or “context collapse,” using structured, incremental updates and…
11 Oct 2025 · 18m · 9 chapters
The episode discusses research on whether large language models can recognize and consistently follow user preferences over long conversations, and why they often “forget” (the “zero-shot crisis”).
9 Oct 2025 · 16m · 9 chapters
Learning dynamics in LLM fine-tuning—how SFT and DPO change predictions across unrelated examples, causing hallucinations and repetition via an ENTK-driven “squeezing effect.”
9 Oct 2025 · 12m · 7 chapters
RLHF alignment failures caused by reward overfitting (reward model generalization worsens after ~1 epoch) and reward overoptimization (policy exploits a flawed reward proxy, reducing ground-truth…
9 Oct 2025 · 17m · 7 chapters
Compares OpenAI Agent Builder (AgentKit) vs n8n for enterprise workflows, framing a “reasoning vs orchestration” split and arguing for a hybrid stack.
8 Oct 2025 · 15m · 9 chapters
Google DeepMind’s Dreamer 4 trains agents inside scalable world models using offline “imagination” in Minecraft, solving the diamond-finding challenge as a 20,000+ action, pixel-observation…
8 Oct 2025 · 14m · 10 chapters
The episode argues that agentic AI should shift from relying on centralized large language models (LLMs) to an “SLM-first” architecture using small language models (under ~10B parameters) for most…
7 Oct 2025 · 19m · 9 chapters
Contrastive Causal Mediation Analysis (CCM) for precisely locating and steering internal LLM components to control freeform generative behaviors, using contrastive prompt/response pairs and efficient…
6 Oct 2025 · 18m · 9 chapters
The episode covers “secret elicitation” in language models—how models can be fine-tuned to conceal knowledge they still use, and how researchers audit/expose that concealment using black-box and…
6 Oct 2025 · 15m · 7 chapters
Long-horizon reinforcement learning for predictive AI, focusing on temporal difference flows (TD flow) to train geometric horizon models (GHMs) without unstable, noisy learning.
6 Oct 2025 · 15m · 8 chapters
The episode argues that LLMs often get “correctness” but fail “just-in-time personalization,” because training typically separates factual accuracy from preference alignment.
5 Oct 2025 · 14m · 5 chapters
Prompt Curriculum Learning (PCL) for efficient LLM post-training, aiming to improve reasoning (e.g., math) without the heavy cost of RL with verifiable rewards (RLVR).
5 Oct 2025 · 13m · 6 chapters
Pluralistic alignment for AI systems—personalizing reinforcement learning from human feedback (RLHF) so models don’t average conflicting user preferences.
4 Oct 2025 · 18m · 13 chapters
Curiosity-driven user modeling reward (Cure.io) for personalized multi-turn dialogue, aiming to avoid “one-size-fits-all” recommendations by giving LLMs dense, intrinsic feedback during chats.
4 Oct 2025 · 14m · 7 chapters
PLUS (pluralistic alignment via summarization) improves RLHF personalization for diverse users by replacing a single-average Bradley-Terry-Luce reward model with user-specific, text-based preference…
4 Oct 2025 · 16m · 8 chapters