Summary: Open-weight models, notably Chinese ones like GLM, Kimi, and Qwen, trail the frontier by roughly six months at a fraction of the price, pushing proprietary providers to cut prices. Enterprise fear has faded since weights can run inside trusted clouds or your own VPC. They win where you need real fine-tuning or LoRA on domain data, privacy (local subtasks routed from a main agent), or edge deployment. “Open” varies (weights only vs data and recipes too). Self-hosting isn’t automatically cheaper than a cheap hosted model. They are also the basis of “sovereign AI”: governments running adapted open models in their own isolated data centers. Choose use cases carefully: conversational answers can reflect different training values.
Key ideas
- Chinese models (GLM, Kimi) “give a phenomenal fight”; proprietary prices are starting to drop (ai-engineering-podcast—why-your-llm-costs-so-much, sharon-dahan).
- Banks once panicked at anything Chinese; now they’re open to it because open weights aren’t executable code (ai-engineering-podcast—why-your-llm-costs-so-much).
- Remaining reluctance is either not enough cost pain or unreasonable fear (e.g. political) (ai-engineering-podcast—why-your-llm-costs-so-much).
- Treat any model, Chinese or not, with the same care: assume it can act wrongly, as security teams assume of humans (ai-engineering-podcast—why-your-llm-costs-so-much).
- monday.com doesn’t call Chinese vendors’ APIs; it runs models via sub-processors (AWS, Google, NVIDIA) under existing security and data-residency contracts, inside its VPC (ai-engineering-podcast—ai-infra-at-scale, dor-cohen).
- Use-case fit: coding output is largely value-neutral; conversation differs (e.g. answers about Taiwan) (ai-engineering-podcast—ai-infra-at-scale).
- US enterprise customers may not want Chinese models, so model choice must stay with the customer, with transparency (ai-engineering-podcast—ai-infra-at-scale).
- “Open” varies: NVIDIA Nemotron released weights, training data, and SFT data, which made continued pre-training reproducible. Licenses ruled out some candidates (commercial use) (explainable—157-training-hebatron).
- Prediction: within 6–12 months more enterprises will fine-tune open-weight models on their own agents’ failure cases (“shift left” into the weights), especially where data is rare (cyber, medical, drug research) (hidden-layers—alice-avi-golan, avi-golan).
- Bria ships its image foundation models with weights, so enterprises can keep training on proprietary data, run on-prem at a capped license cost, and avoid generic outputs (hidden-layers—bria-misha-feinstein, misha-feinstein).
- Open models trail the frontier by roughly six months (with Chinese labs leading open benchmarks); you pick the model size yourself, unlike closed products with routers and system prompts. Kimi K2.5 comes close to Claude Code for coding (hidden-layers—nexar-roni-goldshmidt, roni-goldshmidt).
- Local isn’t automatically cheaper: in Nexar’s video-classification benchmark, Gemini Flash beat a strong Qwen and cost less than the A100 hour needed to host it (hidden-layers—nexar-roni-goldshmidt, roni-goldshmidt).
- Where open wins: real fine-tuning or LoRA on domain data (closed-model tuning barely changes behavior), privacy-sensitive subtasks routed from a main agent via a skill, and possibly future personal agents fine-tuned per user and run locally (hidden-layers—nexar-roni-goldshmidt, roni-goldshmidt).
- Open-weight models now reach frontier size: Kimi at ~1T parameters, DeepSeek V3/V3.2 at ~670B, and DeepSeek V4 at reportedly ~1.4T with notable architectural changes. At that size you have to split them across GPUs (osim-tochna—running-llms-at-scale, mike-erlihson).
- Sovereign AI for defense: build national data centers, install open-source models, and adapt them under your own control, isolated from the internet. Open source is ~6 months behind the frontier, “good enough for 99% of use cases” (micha-stocks—naftali-bennett-economic-interview, naftali-bennett).
Disagreements & open questions
Takeaways
- Benchmark an open-weight model (e.g. GLM, Kimi, Qwen) on your workload for cost savings (ai-engineering-podcast—why-your-llm-costs-so-much)
- Route privacy-sensitive subtasks (e.g. bank statements) to a local open model from your main agent via a skill (hidden-layers—nexar-roni-goldshmidt)
Related
model-selection · llm-inference · ai-guardrails · ai-gateway · hebrew-llms