VQV Signal
According to a post on X by @Thom_Wolf, 18 of the 20 best open models are Mixture-of-Experts (MoE), and dense models are "basically over." The post says Hugging Face spent about 800k GPU-hours on "The Ultimate MoE Training Handbook," which runs to about 70k words with 55 interactive figures. The handbook covers expert counts, all-to-all costs, routers, aux loss versus loss-free bias, upcycling, and serve-time expert pruning.
According to a post on X by @Thom_Wolf, 18 of the 20 best open models are Mixture-of-Experts (MoE), and dense models are "basically over." The post says Hugging Face spent about 800k GPU-hours on "The Ultimate MoE Training Handbook," which runs to about 70k words with 55 interactive figures. The handbook covers expert counts, all-to-all costs, routers, aux loss versus loss-free bias, upcycling, and serve-time expert pruning.
AI-assisted summary based on the listed source.
Thom Wolf says 18 of the 20 best open models are Mixture-of-Experts and that dense is “basically over.” Hugging Face spent about 800k GPU-hours and released “The Ultimate MoE Training Handbook” (~70k words, 55 interactive figures) covering expert counts, all-to-all costs, routers, aux loss vs loss-free bias,...
Teams training or choosing open models may need to weigh MoE architectures more closely, since the post argues dense designs are losing ground. The handbook offers a detailed reference on trade-offs such as routing and communication costs. Watch whether its guidance shows up in future open model releases.
Hardware and robotics watchers may want to track whether this becomes a product, benchmark, or deployment signal.
VQV organizes public signals from inspectable sources. It does not independently verify the underlying report.
Signal Strength reflects source quality, relevance, freshness and evidence. Public Interest helps organize discovery; it is not proof of truth.
VQV surfaced this signal because it is recent, relevant to AI Chips, connected to Scobleizer News Pulse (X).
No login, cookies, social SDKs, or automatic posting.
