gilad-levi is a former child prodigy who finished a Technion CS degree at about 17 and built its first transformers/pre-training course at about 18. He skipped a PhD because academic papers now test ideas on 1–7B models that don’t validate at scale, and he founded Manifold, an Israeli AI lab working on continual learning. LLMs learn very differently from humans: they compress huge unlabeled datasets and hope for generalization (grokking). Humans learn from sparse signals and update with every new fact. Memory and retrieval systems only decide which notes to re-read each morning instead of distilling knowledge as it arrives. He covers pre-training folklore (context-length curricula that break at untrained lengths, multi-head latent attention known long before DeepSeek published it) and why token-space reasoning is wasteful and may not reflect the model’s latent computation. On business: technology is a “business card”, not a moat. Selling tokens with public benchmarks invites one-upping, so build something “good enough” with value beyond the model, as Spotify did. Labs are fighting fires in an arms race, and trillion-dollar IPOs raise questions about public money. Also: Israel’s AI brain drain and the goal of a world-class lab here; advice to parents to step out of the way, and to young people to check they’re climbing the right ladder.
Show: explainable
Concepts: continual-learning · llm-pretraining · llm-reasoning · ai-moats · agent-memory · rag · ai-hype