VQV Signal
According to a post on X by @spheron, the company reported cost figures for Marlin, a 4-bit kernel, running Qwen2.5-32B AWQ on an H200 GPU. The post says Marlin reached 741 tokens per second, against 68 for vLLM's default AWQ kernel, about 10.9x faster. It also says cost per million output tokens fell from $19.61 to $1.80. The post follows an earlier 4-bit kernel benchmark from Spheron, and the figures are Spheron's own claim.
According to a post on X by @spheron, the company reported cost figures for Marlin, a 4-bit kernel, running Qwen2.5-32B AWQ on an H200 GPU. The post says Marlin reached 741 tokens per second, against 68 for vLLM's default AWQ kernel, about 10.9x faster. It also says cost per million output tokens fell from $19.61 to $1.80. The post follows an earlier 4-bit kernel benchmark from Spheron, and the figures are Spheron's own claim.
AI-assisted summary based on the listed source.
Spheron follows its earlier 4-bit kernel bench with hard cost numbers on the same Qwen2.5-32B AWQ/H200 setup: Marlin at 741 tok/s versus vLLM’s default AWQ kernel at 68 (~10.9x), cutting cost per million output tokens from $19.61 to $1.80. A reply links the full GPTQ vs AWQ vs GGUF blog; treat figures as Spheron’s...
Teams comparing 4-bit inference setups for large language models may look at cost per million tokens as well as throughput. These numbers come from a single vendor on one model and one GPU, so buyers would need to test their own workloads before relying on them. The post links a blog comparing GPTQ, AWQ and GGUF, which may provide more context.
VQV organizes public signals from inspectable sources. It does not independently verify the underlying report.
Signal Strength reflects source quality, relevance, freshness and evidence. Public Interest helps organize discovery; it is not proof of truth.
VQV surfaced this signal because it is recent, relevant to LLM Inference, connected to Scobleizer News Pulse (X).
No login, cookies, social SDKs, or automatic posting.
