Summary: Speculation about agents moving from supervised tools to fully autonomous actors. Since late 2025, models sustain much longer tasks, and agents can explore many branches in parallel, so a good verifier can replace a perfect agent. Software is an ideal domain because mistakes are cheap to revert, unlike self-driving cars. Open questions remain: extrapolation beyond training data, continuous learning baked into weights, and whether labs will fund AGI research over monetization. Speculative frontiers include self-funding autonomous businesses and agents hiring humans for physical tasks.
Key ideas
- Intelligence is becoming a commodity that scales with money, hardware, and energy, unlike any previous tech shift (langtalks—63-wake-up).
- Cost of error: in software you can revert; in self-driving you can’t. That’s why autonomy arrives in code first (langtalks—63-wake-up).
- The last mile is hard: 90→95→100% isn’t like 60→90%. LLMs interpolate well but struggle to extrapolate, and eager instruction-following can lead into rabbit holes (langtalks—63-wake-up).
- Parallelism plus verification: agents aren’t limited in time or resources, so they can try many branches and keep what succeeds. That outcome data then trains the next models, a possible self-improvement loop (langtalks—63-wake-up).
- Wanted: models that learn during use and consolidate into weights overnight, instead of clunky markdown memory (langtalks—63-wake-up).
- Skepticism: labs may prioritize business exploitation over continuous-learning research, despite AGI rhetoric (langtalks—63-wake-up).
- Speculative: an agent with a crypto wallet paying for its own VPS, earning money to “survive”, spawning replicas; services where agents hire humans for physical-world tasks (e.g. building data centers) (langtalks—63-wake-up).
- Humans become the bottleneck. The goal is agents that predict our feedback and keep working while we sleep (langtalks—63-wake-up).
- The length of tasks agents can complete is growing exponentially (minutes for GPT-4 to hours for Opus 4.5); weeks- or months-long runs may follow, driven as much by orchestration frameworks as by models (langtalks—62-ai-rd-rollout).
- “Alignment engineering” (tooling for alignment between humans and AI) may be the next hype after context engineering (langtalks—62-ai-rd-rollout).
- The missing piece is continuous, goal-directed learning (learning while acting, as humans do to survive); current long-term memory is heuristic feature engineering around static models (langtalks—60-brain-memory).
Disagreements & open questions
- Will frontier labs pursue continuous learning/AGI or focus on monetizing current models (langtalks—63-wake-up)?
- Files or weights? Itamar Friedman prefers natural-language memory files for now because you can read and veto what was learned (langtalks—57-memory, itamar-friedman); the LangTalks hosts want models that learn during use and consolidate into weights overnight instead of “clunky markdown memory” (langtalks—63-wake-up), and Gilad Levi argues file-based memory is ill-posed (explainable—164-gilad-levi-continual-learning, gilad-levi).
Takeaways
Related
future-of-software-engineering · harness-engineering · ai-hype · human-vs-ai-memory