Week in review: Oct 25 – Nov 1
Local AI keeps shrinking — a 74B MoE on a 24GB GPU and a 763B model claimed on 12GB VRAM — while identical-harness bake-offs tighten verification and Pivot to AI dissects Anthropic's numbers-free IPO and the Chinese AI correction.
A moderate week: fifteen items across five rollup days (Oct 25–28 and Oct 30), and they sort cleanly into two camps — the local-AI capability grind and the AI-money accountability beat.
The local camp was dominated by compression. Nvidia’s Nemotron Bolt squeezes a 74B mixture-of-experts into about 22 GB so it runs full-context on a 24 GB GPU; DeepSeek v4.1 Flash’s 763 billion parameters reportedly run on 12 GB of VRAM given enough RAM and Das Lab’s GSQ RCO quantization; and Ternary Bonsai 2’s 7.2 GB PQ2 quant retained enough of a Qwen 3.8 27B’s ability to build a working game asset editor inside 12 GB. Distillation is the other lever — one experiment pushed a 4B Qwen toward 27B territory on Kaggle-style analysis after RL plateaued at 58.4%. Alongside the shrinking, the bake-offs got more rigorous: the Kimi K3.1 Pro vs Claude Opus 5.5 comparison ran both models through the same 13 long-horizon tasks in one harness with identical prompts and grading scripts, and it found Kimi genuinely competitive — the same verification culture showing up in the Qwen Swift and Flash Next vs Coder comparisons.
The money beat was all about numbers that don’t add up. Anthropic filed a preliminary S-1 for a proposed Nasdaq IPO at a reported ~$1.5 trillion valuation that Pivot to AI called a “fill-in-the-blanks” prospectus because the key financials are missing; David Gerard framed the Chinese AI stock crash as a cheering correction of speculative excess rather than the end of AI; and Ed Zitron sat down with Paris Marx to walk through the social and environmental bill coming due on the data-center buildout.
Notable items
- Kimi K3.1 Pro and Opus 5.5 Made the Same 13 Apps (Raw Results) — identical harness, prompts, and grading scripts make this one of the fairest agentic coding bake-offs yet, with Kimi surprisingly competitive.
- Running DeepSeek v4.1 Flash on local hardware — a 763B-parameter model claimed to run on 12GB of VRAM with enough RAM keeps pushing what “local” can mean.
- Nemotron Bolt: first look and test — a 74B MoE compressed to about 22 GB runs with a full context window on a single 24 GB GPU.
- Building a game asset editor in 12GB VRAM — Ternary Bonsai 2’s ~7.2 GB quantization retains most of a Qwen 3.8 27B’s capability on modest hardware.
- I Trained Qwen 4B with Distillation to Catch Qwen 27B — a three-way comparison of distillation methods after reinforcement learning plateaued the small model at 58.4%.
- NCM: The End of LLMs? Not So Fast (Local Tests) — Stanford’s Next-Concept Modeling sketches a whole answer then refines it in a few passes instead of emitting tokens one at a time.
- Qwen 3.8 Flash Next vs Flash Next Coder — Flash Next cracking 60+ tokens/sec on a dual-GPU box, and where the Coder variant actually earns its keep.
- The Truth About Data Centers w/ Paris Marx | Better Offline — the social, environmental, and political bill coming due on the AI-driven data-center buildout.
- The Chinese AI crash is not the end, but it is cheering — Gerard argues the burst is a welcome correction of speculative excess, not a verdict on AI itself.
- Anthropic files its ‘IPO’ — but key numbers are missing — a “fill-in-the-blanks” S-1 targeting a reported ~$1.5 trillion valuation with the key financials left out.