VQV Signal

CREATOR TOOLS WATCH TECHNICAL

Work-in-progress notebook shows hand-built SwiGLU neurons changing model behavior

According to a post on X by @voooooogel, a work-in-progress notebook on vgel.me shows how to insert single SwiGLU neurons into a transformer by hand, with no optimizer. The method uses word-vector intuitions to set a read (gate/up) vector and a write (down) vector. Examples on Qwen3.6-27B include one neuron that makes the model answer rude users rudely and another that decensors answers. The snippet covers only part of an unfinished notebook, so the full method and results are not described here.

Source: Scobleizer News Pulse (X) · x.com Found via Robert Scoble’s Scobleizer News Pulse · a post on X, not independently verified Published 2026-11-21T05:50:00+00:00 Detected 2026-11-21T20:01:37+00:00
View original source

According to a post on X by @voooooogel, a work-in-progress notebook on vgel.me shows how to insert single SwiGLU neurons into a transformer by hand, with no optimizer. The method uses word-vector intuitions to set a read (gate/up) vector and a write (down) vector. Examples on Qwen3.6-27B include one neuron that makes the model answer rude users rudely and another that decensors answers. The snippet covers only part of an unfinished notebook, so the full method and results are not described here.

AI-assisted summary based on the listed source.

@voooooogel posted a work-in-progress vgel.me notebook on inserting single SwiGLU neurons into a transformer by hand, with no optimizer, using word-vector intuitions to set a read (gate/up) vector and a write (down) vector. Examples on Qwen3.6-27B include a neuron that makes the model answer rude users rudely and...

For developers working with open-weight models, this suggests a possible way to change specific behaviors by editing individual neurons without running an optimizer. The examples are limited to the two described in the post, and the notebook is still in progress. Watch whether the full notebook shows results that hold up across other prompts and models.

Signal Strength 78% Technical label WATCH Public Interest 21 Category CREATOR TOOLS Reader Depth TECHNICAL

Signal Strength reflects source quality, relevance, freshness and evidence. Public Interest helps organize discovery; it is not proof of truth.

Public Interest components
Recognizable Entity Score 0 Practical Impact Score 8 Novelty Interest Score 70 Consequence Score 10 Curiosity Score 0 Shareability Score 42

VQV surfaced this signal because it is recent, relevant to Open Source LLMs, connected to Scobleizer News Pulse (X).