Summary
According to a post on X by @rohanpaul_ai, a Harvard and MIT paper finds that reasoning models such as Qwen3-32B and OLMo-3.1-32B put about 99% of first-token probability on "Okay" when reasoning. The post says this habit helps explain why these models struggle to follow instructions about how to reason. Banning "Okay" and "Alright" along with one worked example raised compliance by 34 to 41 percentage points. The excerpt also mentions tasks about hiding information behaving like a separate skill across 12 models, but it is cut off before giving details.
AI-assisted summary based on the listed source.
What happened
The paper finds models like Qwen3-32B and OLMo-3.1-32B put about 99% of first-token probability on "Okay" when reasoning, which helps explain why they struggle to follow instructions about how to reason. Banning "Okay" and "Alright" plus one worked example raised compliance by 34 to 41 percentage points, while...
Why it matters
Developers who depend on controlling how open source LLMs reason may need to test whether reasoning-style instructions are actually followed. The result points to a specific, measurable behavior that can be studied and tested. Readers should check the full paper for its complete method and results before relying on the figures.
Signal Intelligence
Signal Strength 90%
Technical label WATCH
Public Interest 28
Category RESEARCH
Reader Depth TECHNICAL
Signal Strength reflects source quality, relevance, freshness and evidence. Public Interest helps organize discovery; it is not proof of truth.
Public Interest components
Recognizable Entity Score 0
Practical Impact Score 8
Novelty Interest Score 94
Consequence Score 18
Curiosity Score 0
Shareability Score 46
Why this is here
VQV surfaced this signal because it is recent, relevant to Open Source LLMs, connected to Scobleizer News Pulse (X).
