We fine-tuned a 4B to outrun models five times its size

What we changed

Smaller “thinking” models often imitate long chain-of-thought without the calibration of a frontier teacher: they meander, second-guess in public, and bury the actual algorithm under performative self-dialogue. We distilled with traces aligned to Qwen3.6-plus so the student learns when to be brief, when to decompose, and when to state invariants up front.

The result is LeemerLabs/Qwen3-4B-Qwen3.6-plus-Reasoning-Distilled — same backbone family as the base checkpoint, with post-training that rewards concise structure over exploratory volume. In practice that means fewer “Hmm / Wait / Actually” loops and more problem → approach → steps → complexity → pseudocode.

Multi-domain benchmark

We evaluate on a multi-domain reasoning suite (same methodology as LeemerLabs/Multi-Domain-Reasoning-Benchmark). The chart below compares our distilled 4B against the Qwen3-4B-Thinking-2507 base on success rate per task class.

Largest relative lifts for the distilled model show up in Python code analysis & debugging and planning / optimisation — the kinds of problems where a clean plan matters more than vocabulary size. Trade-offs remain (for example, SQL in this run still favours the base checkpoint); we publish the numbers so you can map them to your own stack.

Reasoning style: base vs. distilled

Base (Qwen3-4B-Thinking). Stream-of-consciousness and exploratory. The model often auditions several interpretations, rereads the prompt mid-trace, and can loop on self-correction before converging. Useful for research-style exploration — noisy as a default production interface.

Distilled (LeemerLabs, Qwen3.6-plus–aligned). Report-shaped: it separates input, output, constraints early, names an algorithmic plan (for graph-style tasks, e.g. state-space search), and holds a steadier line from reasoning to pseudocode. Same small backbone, behaviour closer to an engineering handoff than a live stream of consciousness.

The 4B on a free endpoint — then something much larger

We are wiring the distilled 4B to the public, free OpenAI-compatible gateway under the model id leemerlabs/qwen3-4b-power. Same EU residency story as the rest of LeemerLabs: requests stay on our infrastructure in Europe. When the slot is live, it will show up in the same console flow as the other public models; until then, request access and we will add you to the allowlist.

The 4B is the opening move. In parallel we are fine-tuning from a ~27B open-weight family, and we are not stopping there — the pipeline also targets an even stronger open-source teacher so we can distil a model that feels like a local, free-inference beast: serious reasoning depth without shipping prompts overseas or paying Silicon Valley by the token. Training and eval loops run on EU metal in Ireland.

The distilled weights are a LeemerLabs build — not a relabel of a public small-model drop. If you want to fine-tune from the same family or run evals in our environment, start with Foundry or request access.