Summary: AI tooling churns weekly, and every week there’s a “best model in the world”. Chasing each one wastes time; most don’t survive, so some practitioners wait a month before adopting anything. Others say the answer to feeling behind is a quick hands-on POC, since most tools share the same principles. And when building products: anything someone can recreate with a prompt in 10 minutes has no value.
Key ideas
- “If it survived the month, then talk about it”: most tools don’t survive (e.g. a once-viral open-source agent that vanished) (ai-engineering-podcast—why-your-llm-costs-so-much, sharon-dahan).
- Companies need stable processes; chasing the next thing isn’t right. If it’s good, it will stay, and you won’t miss it (ai-engineering-podcast—why-your-llm-costs-so-much).
- Don’t build what can be built in 10 minutes with a prompt (ai-engineering-podcast—why-your-llm-costs-so-much).
- Benchmarks “endanger humanity every week”, then the model can’t do your task (ai-engineering-podcast—why-your-llm-costs-so-much).
- Jev was overhyped: a classifier, not an LLM replacement. Benchmark on your own data and A/B test before switching (langtalks—74-jev).
- Counterpoint to waiting: Wonderful builds features that are “half-broken” on current models, betting model progress makes them work, rather than adding restrictions around today’s limits (ignore-instructions—24-wonderful-road-to-100m).
- Example of churn: a colleague told Levi to switch to Codex (“Claude Code is dead”), then back to Claude Code a week later. Tools like alphaXiv feel five years old a week after launch; you’re expected to be a year-long expert in something released this week (explainable—164-gilad-levi-continual-learning, gilad-levi).
- Gartner-style peak of inflated expectations: there may be a dip, but the long-tail compounding effect is large and shows up in unexpected places (ignore-instructions—26-ai-cyber-pavel-gurevich, pavel-gurevich).
- Veteran’s rule: wait about two weeks before chasing a new tool; if it’s still exciting, try it. Remember the 80/20 gap between a hackathon POC and production. Like the client-side framework churn of the past, nobody gets fired for learning a trend a month later (osim-tochna—ai-in-production-reality-vs-imagination).
- It’s harder to decide you don’t need a new technology than to say “let’s try it”; a team running production agents asks that question constantly (startup-for-startup—370-agentos-month-to-day, moshe-krispin).
- “Don’t be afraid”: many engineers feel behind and freeze. Build a small POC to grasp the principle; most products are similar, so understanding one explains the next (startup-for-startup—370-agentos-month-to-day, moshe-krispin).
Disagreements & open questions
- Wait or try now? Sharon Dahan’s rule is to talk about a tool only once it has survived a month, and an Osim Tochna guest waits about two weeks (ai-engineering-podcast—why-your-llm-costs-so-much, sharon-dahan, osim-tochna—ai-in-production-reality-vs-imagination); Roy Mann urges trying things hands-on because month-old verdicts are already history (startup-for-startup—362-agent-architecture-fit, roy-mann), and Moshe Krispin says to build a quick POC rather than fear the churn (startup-for-startup—370-agentos-month-to-day, moshe-krispin).
Takeaways
- Wait a month before adopting a new AI tool or model (ai-engineering-podcast—why-your-llm-costs-so-much)
- Sanity-check product ideas: could a prompt recreate this in 10 minutes? (ai-engineering-podcast—why-your-llm-costs-so-much)
- When a new tool makes you feel behind, spend an hour on a tiny POC to learn its core idea (startup-for-startup—370-agentos-month-to-day)