Attention — Oct 30, 2026
What caught my attention on Oct 30 (2 items).
I Trained Qwen 4B with Distillation to Catch Qwen 27B
Prompt Engineering (Article:11:03) - Published 2026-10-28 18:00 - Discovered 2026-10-30 11:03
The video tests whether a 27B Qwen model can distill its abilities into a 4B Qwen model on Kaggle-style data-analysis tasks in a sandbox, after reinforcement learning previously raised the small model from 52% to 58.4% and then plateaued. The author compares three distillation methods: training on the teacher’s correct transcripts, training on the teacher’s token probabilities, and on-policy disti…
Anthropic files its ‘IPO’ — but key numbers are missing
Pivot to AI (Article:11:05) - Published 2026-10-26 18:00 - Discovered 2026-10-30 11:05
This Pivot to AI episode discusses Anthropic’s preliminary S-1 filing for a proposed Nasdaq IPO, which the hosts describe as a “fill-in-the-blanks” prospectus because many key financial numbers are missing. Anthropic reportedly plans to go public at a valuation of about $1.5 trillion and hopes to raise around $100 billion. The video emphasizes that the filing is heavily promotional, framing Anthro…