Summary
According to a post on X by @SnorkelAI, Snorkel's research team says its Qwen3.8-27B model, trained on 1,105 expert-built GDPval+ tasks, rose from 31.9% to 40.7% pass@1 on OpenAI's GDPval gold set. The team says standard GRPO barely moved the score, while a binary-gated GRPO variant learned from failed attempts. The snippet describes these results as the company's own findings and does not mention independent verification.
AI-assisted summary based on the listed source.
What happened
Snorkel's research team says Qwen3.8-27B trained on 1,105 expert-built GDPval+ tasks rose from 31.9% to 40.7% pass@1 on OpenAI's GDPval gold set. It says standard GRPO barely moved, while a binary-gated GRPO variant learned from failed attempts. Why it matters: Small open models may close gaps on real work tasks...
Why it matters
Teams building on open models for real work tasks may find that targeted expert data can narrow the gap on those tasks, if the reported results hold up. Anyone evaluating open models should watch for the full methods and whether outside groups can reproduce the GDPval gains. For now, the results remain one company's reported findings.
Signal Intelligence
Signal Strength 82%
Technical label WATCH
Public Interest 21
Category RESEARCH
Reader Depth TECHNICAL
Signal Strength reflects source quality, relevance, freshness and evidence. Public Interest helps organize discovery; it is not proof of truth.
Public Interest components
Recognizable Entity Score 0
Practical Impact Score 8
Novelty Interest Score 70
Consequence Score 10
Curiosity Score 0
Shareability Score 42
Why this is here
VQV surfaced this signal because it is recent, relevant to Open Source LLMs, connected to Scobleizer News Pulse (X).