VQV Signal

RESEARCH WATCH TECHNICAL

Snorkel reports a GDPval score gain for a 27B open model after expert-task training

According to a post on X by @SnorkelAI, Snorkel's research team says its Qwen3.8-27B model, trained on 1,105 expert-built GDPval+ tasks, rose from 31.9% to 40.7% pass@1 on OpenAI's GDPval gold set. The team says standard GRPO barely moved the score, while a binary-gated GRPO variant learned from failed attempts. The snippet describes these results as the company's own findings and does not mention independent verification.

Source: Scobleizer News Pulse (X) · x.com Found via Robert Scoble’s Scobleizer News Pulse · a post on X, not independently verified Published 2026-11-19T17:45:00+00:00 Detected 2026-11-20T00:02:13+00:00
View original source

According to a post on X by @SnorkelAI, Snorkel's research team says its Qwen3.8-27B model, trained on 1,105 expert-built GDPval+ tasks, rose from 31.9% to 40.7% pass@1 on OpenAI's GDPval gold set. The team says standard GRPO barely moved the score, while a binary-gated GRPO variant learned from failed attempts. The snippet describes these results as the company's own findings and does not mention independent verification.

AI-assisted summary based on the listed source.

Snorkel's research team says Qwen3.8-27B trained on 1,105 expert-built GDPval+ tasks rose from 31.9% to 40.7% pass@1 on OpenAI's GDPval gold set. It says standard GRPO barely moved, while a binary-gated GRPO variant learned from failed attempts. Why it matters: Small open models may close gaps on real work tasks...

Teams building on open models for real work tasks may find that targeted expert data can narrow the gap on those tasks, if the reported results hold up. Anyone evaluating open models should watch for the full methods and whether outside groups can reproduce the GDPval gains. For now, the results remain one company's reported findings.

Signal Strength 82% Technical label WATCH Public Interest 21 Category RESEARCH Reader Depth TECHNICAL

Signal Strength reflects source quality, relevance, freshness and evidence. Public Interest helps organize discovery; it is not proof of truth.

Public Interest components
Recognizable Entity Score 0 Practical Impact Score 8 Novelty Interest Score 70 Consequence Score 10 Curiosity Score 0 Shareability Score 42

VQV surfaced this signal because it is recent, relevant to Open Source LLMs, connected to Scobleizer News Pulse (X).